Have you ever forgotten to shut down your test instances that you just used for few days and got a surprise when the monthly AWS bill came? Do you want to stop your development instances automatically at night and weekends when you don’t use them? If your answers are yes, you will want to continue reading this post.
It is fairly easy to set up a schedule to start and stop your EC2 instances automatically to control your EC2 usage. There are quite a few 3rd party providers offering this service. If you search for “EC2 schedule” or “auto start stop EC2″ in google or AWS forums, you will be able to see what is available. I list few of them in alphabetical order for your reference in case you are interested:
Of course, there is a monthly subscription fee associated with these 3rd party services, and you also need to give out your access key and secret key to the 3rd party service provider. These service providers offer more feature than just auto start and stop EC2 instances. It is probably worth using them if you are planning to use other features as well. Otherwise, it may be hard to justify when you can do it on your own without revealing your credentials to a 3rd party provider.
In fact, AWS has provided all the APIs you need to manipulate EC2 instances. That’s how these 3rd party service providers are able to manage AWS resources. It is indeed pretty straight forward. All you really have to do is to put together a simple Python script to leverage these APIs with Boto, which are well documented and supported.
In this post, I will show you how to use a micro instance with an IAM role to start and stop your EC2 instances automatically within a region. Here are the 3 simple steps to get started:
- Create an IAM role
- Provision a micro EC2 instance
- Tag your instances
STEP 1: Create an IAM Role
On the IAM Management Console, go to Roles and click Create New Roles to open a wizard to start the process to create a new role. In the screenshots below, I called the IAM role as ec2-operator. You can call it whatever you want as long as you can identify it.
Type a self-explanatory name to the Role Name field. It is used for identifying your IAM role. You will need to use the name later when you create a micro EC2 instance.
On the next screen, select AWS Service Roles and then Amazon EC2.
On the next screen, select Custom Policy.
Type an arbitrary name to the Policy Name field.
Paste the following JSON block to the Policy Document field to indicate the IAM role is allowed to perform describe instances, start instances and stop instances action.
Review the role information and click Create Role to finish. The new role should be created immediately.
STEP 2: Provision a micro EC2 instance
I recommend to use a micro EC2 instance. I think it is sufficient to handle what we need here to set up a cron job to run a Python script to process auto-start and auto-stop requests, but feel free to use a different instance type.
On the EC2 Management Console, click Launch Instance to start the process to provision a micro EC2 instance. Let’s call this instance as an EC2 operator.
I recommend to select the latest Amazon Linux AMI, which has AWS tools, python and boto pre-installed.
You can pretty much take the default value for each step except on Step 3: Configure Instance Details and Step 6: Configure Security Group.
On the Configure Instance Details screen, you need to make sure you select the IAM role you created and assign it to the instance. In my case, I selected ec2-operator as the IAM role. You will not be able to change the IAM role for the instance once it is launched.
You can cut and past the following shell script into the User Data field. It will install few Python libraries and modules, create a Python script, and set up a cron job to execute it every 5 minutes. The shell script is set to operate instances for us-east-1. If you need to operate instances for a different region, you can replace us-east-1 with let’s us-west-1 or another region. It is case-sensitive.
On the Configure Security Group screen, make sure you are not open the instance to the world (0.0.0.0/0) for the security group. In fact, you can get rid of the security rule. You don’t really need to SSH to the instance.
STEP 3: Tag your instances
The last step is go to EC2 Dashboard Console to tag your instances that you want to start and stop automatically at certain time. The EC2 operator instance looks for the auto:start and auto:stop tags to determine when to start and stop instances. The name of the tags are case-sensitive. The value of the tags should be in a cron format. For instance, 0 14 * * * is 2 pm everyday. The time is based on the OS time of the EC2 operator instance. If you have not changed the default time zone of the EC2 operator instance, it should be in UTC. It will ignore any instances that do not have the tags set or the format is invalid. The auto:start tag holds the start schedule, and the auto:stop tag holds the stop schedule. Do not tag the EC2 operator instance, otherwise it will get stopped by itself.
The script may not start or stop instances at the requested time exactly. For the start operation, it will start the instance between 30 minutes prior to the start time and the requested time. For the stop operation, it will stop the instance between the requested time and 30 minutes after the requested time. The EC2 operator instance is set to check every 5 minutes. It will process any auto-start and auto-stop requests few times if they have not started or stopped during the 30-minute window in case the operation does not go well.
I have been using the script for a while, and it has been performing pretty well. I am able to get all my development instances stopped during non-business hours. I am even able to check the operations with CloudTrail to audit if the instances were indeed started and stopped within few minutes of the requested time.