Elastic Compute Cloud: EC2 (Intro to AWS Series) Part 2
Welcome to Part 2 of my EC2 section of my Intro to AWS series. This is a continuation of part 1, which you can read here: https://blog.johnwalker.tech/elastic-compute-cloud-ec2-intro-to-aws-series.
In this post, I will continue with some topics I mentioned at the end of the last post. I will have a Part 3 coming soon.
Amazon Machine Images
AWS provides a way of taking a snapshot of a running instance so you can restore it at a later time, or use it as the basis of an Auto Scaling Group (covered later in this post). These Amazon Machine Images (or AMIs) are a very useful way to both start new instances from a predictable baseline and to use as part of a backup process.
You can create your own AMIs easily from a running instance from the console just by right clicking on an instance and under images, choosing 'create image':
AWS also provides AMIs for you to launch instances from. You can see the most common of these when you go to launch instance:
You can also launch instances that have been pre-configured for various tasks. There are ones with Python and Django pre-installed, ECS optimised images, Windows with .Net and many more.
AWS also keep there images up to date and on a regular schedule provide an update version with the latest patches and security updates installed so you can launch a new instance from them.
AMIs are also a key part of the backup process as you capture all the current settings and installations of the machine, with an associated snapshot for the EBS volumes attached (more on those later), which means you can restore to a specific point in time if something goes wrong with your machine. These backups can be done manually or through AWS backup.
You can also build your own AMIs using the EC2 Image Builder, which allows you to create an automated pipeline to create and update images with your own code, and combined them into an AMI ready for use. I've covered EC2 Image Builder here: https://blog.johnwalker.tech/ec2-image-builder.
AWS has a large number of instance types that can be used for various purposes. Some of them are more memory orientied, some CPU oriented, some with AWS's own processors, some AMD, some Intel.
I've found that the best way to figure out which instance type you need initially is to use the EC2 Instances site:
Once you have narrowed down how many cores or memory you need you can look at the various prices and choose a relevant instance type.
Once you have used the instance, you can then use something like AWS Compute Optimizer, which will tell you if your instances are over or under provisioned so you can right size your instances for your use case.
Networking & Security
This section will cover some of the remaining Networking and Security topics I didn't cover in Part 1. I would really recommend reading the part on Security Groups in Part 1 in addition to this section.
Key pairs are the way that AWS provides you with an SSH key to be able to connect to your instance. There are a number of other ways to connect and authenticate such as Active Directory and Session Manager.
One of the most common ways is to provide a Key Pair. This is either one you create during the process of luanching an instance, or a precreated one. What happens is a public and private key pair combination is created and the public key is put on the server as it is launched.
What this means is that the machine can encrypt data using the public half of the key that can only be decrypted and read by someone holding the private half of the key. That way the instance can encrypt its end of an SSH connection and you can only respond correctly if you have specified the right private key.
For most use cases of EC2 you wont likely need to create an Elastic Network Interface (ENI) as part of the EC2 creation process will create one for you, they are generally needed when you need a machine with multiple IP addresses or you need a faster lower latency network connection, which is where the Elastic Fabric Adapter comes in, it is useful for use cases such as Machine Learning or High Performance Computing.
Within AWS you can create your own IP that is taken from AWS's pool of IP addresses or you can bring your own IP over to AWS if you can transfer it.
Elastic IPs are able to be created during the EC2 creation process, which gives your instance a public facing IP Address. You can also detach an Elastic IP from an instance and attach it to another, either through the console or programatically using the Command Line (CLI) or one of the AWS Software Development Kits (SDKs).
This is useful if you have your IP whitelisted by a third party, meaning you can change the underlying instance without having to change your IP.
A word of warning though, Elastic IPs are generally free, but if you leave them detached they will cost $0.01/hr for unused ones. In addition you can remap an Elastic IP up to 100 times a month, at which point it starts costing $0.10 per remap. This can really ramp up costs, especially if you do it programatically and accidentally detach and reattach more times than you need, you can quite quickly go over that 100 remap limit.
AWS provides a number of storage options depending on your use case. The most common one to use with EC2 is the Elastic Block Storage or EBS.
Elastic Block Storage (EBSVolumes)
EBS is the equivalent of a hard drive attached to a physical computer. They come in different types and performance, from the 2 types of SSD (Solid State Drives) to the Magnetic Drive type.
GP2 is the standard version an SSD drive and the one you will most commonly chose. You can choose the size from 1GB up to 16TB. The speed of the disk is determined by its IOPS, which is the count of how many Input/Output operations per second the drive is capable of, which determines how fast it can read and write data.
GP2 gets 3 IOPS per GB of data, with burst capacity up to 3000 IOPS. This means you can do up to 3000 operations every second from a base of 3 per second for 1 GB.
If you need more speed or guaranteed (not burstable) speed, the the Provisioned IOPS disks are what you will use. The current generation of Provisioned IOPS SSD based volumes is the io2 type. It provides a variety of sizes from 4GB up to 16TB. You can also choose how fast access is to the drives, by choosing different IOPS levels.
Provisioned IOPS have a minimum speed of 100 IOPS, and a max of 64000.
When choosing your IOPS speed you can gave from 50 IOPS per GB up to 64000.
io1 is the previous generation SSD based volume and it is the same price as io2, but not quite as good performance. The allowable figures are the same, with a min of 4GB and max of 16TB, with IOPS ranging from 100 to 64000.
However, io2 is more durable and you can support up to 500 IOPS per GB, 10 times the allowance for io1.
Unless you have a specific need for io1 or you have existing drives that you don't want to change, I would recommend sticking with the newer io2.
With EBS volumes, you can easily take a snapshot of the data currently on the disk and store this as a backup. These can be stored in s3 and automatically moved to S3 Glacier or deleted as you need.
These snapshots can be used to restore the data by spinning up a new instance and attaching the EBS Volume, or by directly replacing the volume attached ot a running instance.
More to come in Part 3, have decided to split these posts up a bit more to make them easier to cover everything.
- Load Balancing
- Load Balancers
- Target Groups
- Launch Configurations
- Auto Scaling
- More Advanced Topics
- Spot Instances
- Capacity Reservations
- Lifecycle Manager
- Placement Groups