OCI – Instance in Stopping State for what Feels like FOREVER

TLDR – If your instance is stuck stopping from a web console initiated stop action It’s because restarts can take up to 15 minutes

Why does my instance take 15 minutes to restart

According to the CLI reference document here, OCI has hard stop and softstop actions for instances. From the Web console the only exposed actions are the soft actions, SOFTSTOP and SOFTRESET. These are the safest actions with the least chance of injuring your instances. The soft actions send a request to the instance operating system once every 30 seconds for a total of 30 attempts before giving up the graceful option and cycling power to the instance with a hard stop command.

Screenshot 2020-05-25 at 13.08.58.png

30 attempts, every 30 seconds = 900 seconds or 15 minutes. The instance operating system not reacting to a graceful stop request is most likely why your instance some times takes around 15 minutes to restart. On a Windows instance this is normally caused by a dialog box telling you something is preventing shutdown. Maybe you were working on something in Notepad and it refused to close until you saved which blocked shutdown. Other reasons could be a crash in the OS preventing the OS from gracefully shutting down – Loads of reason, check your logs they might help you to understand what is going on the instance during a shutdown attempt.

Basilcy — the OCI service is doing everything it can for 15 minutes attempting to prevent data corruption and issues in your instance that can be caused by an improper shutdown.

NOTE — If your instance is in a stopping state for over 20 minutes please open a support indicent with OCI support asking for assistance, and provide the instance OCID in your support case.

Hard stop an instance

If the risk of Operating system is not a concern to you, and you want to restart or stop your instance immediately you can use API or CLI to initiate a hard stop / power cycle of your instance. The following commands are example CLI commands to graceful SOFTSTOP or hard stop STOP an instance.

oci compute instance action --action SOFTSTOP --instance-id ocid1.instance.oc1.eu-frankfurt-1.antheljrsdy6wiqcnqplrk3inniwkcelbdem3k4ot7nztcga5z4mpnrs4tza
oci compute instance action --action STOP --instance-id ocid1.instance.oc1.eu-frankfurt-1.antheljrsdy6wiqcnqplrk3inniwkcelbdem3k4ot7nztcga5z4mpnrs4tza

 

Screenshot 2020-05-25 at 13.11.44

The avalable verbs for the instance action command which you can call from the API or the CLI are as follows (borrowed directly from the documentation):

  • START – Powers on the instance.
  • STOP – Powers off the instance.
  • RESET – Powers off the instance and then powers it back on.
  • SOFTSTOP – Gracefully shuts down the instance by sending a shutdown command to the operating system. If the applications that run on the instance take a long time to shut down, they could be improperly stopped, resulting in data corruption. To avoid this, shut down the instance using the commands available in the OS before you softstop the instance.
  • SOFTRESET – Gracefully reboots the instance by sending a shutdown command to the operating system, and then powers the instance back on.

Minimum Permissions Policy

Slightly related, permissions. I had a customer open a case to me recently asking for a policy with permission limited to allow only start, or stop actions for instances. He did not want the group to be able to modify any other settings. To satisfy his request I provided a policy limited to only {INSTANCE_POWER_ACTIONS} which is a permissions group that covers the above, start, stop, and reset verbs.

Allow group GROUPNAME {INSTANCE_POWER_ACTIONS} in tenancy

.
Screenshot 2020-05-25 at 13.10.18.png

Leave a Reply