AWS Solution Architect(Associate) - Topic 2: Elastic Compute Cloud (Amazon EC2)

Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides secure, resizable compute capacity in the cloud. It is designed to make web-scale cloud computing easier for developers.

Amazon EC2’s simple web service interface allows you to obtain and configure capacity with minimal friction. It provides you with complete control of your computing resources and lets you run on Amazon’s proven computing environment.

[toc]

Elastic Compute Cloud

Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides secure, resizable compute capacity in the cloud.

Amazon EC2 reduces the time required to obtain and boot new server instances to minutes, allowing you to quickly scale capacity, both up and down, as your computing requirements change.

EC2 Pricing Models

On Demand

Allows you to pay a fixed rate by the hours (or by the second) with no commitment.

• Users want the low cost and flexibility of EC2 without any up-front payment or long-term commitment.
• Application with short-term, spiky, or unpredictable workloads that cannot be interrupted
• Applications being developed or tested on Amazon EC2 for the first time.

Reserved

Provides you with a capacity reservation, and offer a significant discount on the hourly charge for an instance.

• Applications with steady state or predicable usage
• Applications that require reserved capacity
• Users able to make up-front payments to reduce their total computing costs even further.
• Reserved Pricing Types
  • Standard Reserved Instances: These offer up to 75% off on demand instances.
  • Convertible Reserved Instances: It allows you to change between the different instance types. These offer up to 54% off on demand capacity to change the attributes of the RI as long as the exchange results in the creation of Reserved Instances of equal or greater value.
  • Scheduled Reserved Instances: These are available to launch within the time windows you reserve.

Spot

Enables you to bid whatever price you want for instance capacity, providing for even greater savings if your applications have flexible start and end times.

• Applications that have flexible start and end times.
• Applications that are only feasible at very low compute prices.
• Users with urgent computing needs for large amounts of additional capacity.

Dedicated Hosts

An Amazon EC2 Dedicated Host is a physical server with EC2 instance capacity fully dedicated to your use. Dedicated Hosts can help you reduce costs by allowing you to use your existing server-bound software licenses.

• Useful for regulatory requirements that may not support multi-tenant virtualization.(It might be the government says that you cannot support multi tenant virtualization)
• Grateful for licensing which does not support multi-tenancy or cloud deployment.(If you’ve got some really harsh Oracle licensing)
• Can be purchased On-Demand(hourly).
• Can be purchased as a Reservation for up to 70% off the On-demand price.

Security Groups

• All Inbound traffic is blocked by default.
• All Outbound traffic is allowed.
• Changes to Security Groups take effect immediately.
• You can have any number of EC2 instances within a security group.
• You can have multiple security groups attached to EC2 Instances.
• Security Groups are STATEFUL. We don’t have to change inbound and outbound ports if you enable something on the inbound, outbound is enabled automatically for that port.
• You cannot block specific IP addresses using Security Groups, instead use Network Access Control Lists(VPC Section).
• You can specify allow rules, but not deny rules. Because by default, you deny everything.

EBS

Amazon Elastic Block Store (EBS) provides persistent block storage volumes for use with Amazon EC2 instances in the AWS Cloud.

Each Amazon EBS volume is automatically replicated within its Availability Zone to protect you from component failure, offering high availability and durability.

5 Different Types of EBS Storage

Volume Type	Description	Use Case	API Name	Volume size	Max IOPS Volume
General Purpose (SSD)	balances price and performance for a wide variety of transaction workloads	Most Work Loads	gp2	1 GiB - 16 TiB	16000
Provisioned IOPS (SSD)	Highest performance designed for mission-critical applications	Databases	io1	4 GiB - 16 TiB	64000
Throughput Optimized Hard Disk Drive(HDD)	Low cost HDD for frequently accessed, throughput-intensive workloads.	Big Data & Data Warehouses	st1	500 GiB - 16 TiB	500
Cold Hard Disk Drive(HDD)	Lowest cost HDD for less frequently accessed workload	File Servers	sc1	500 GiB - 16 TiB	250
EBS Magnetic	Precious generation HDD	Workloads where data is infrequently accessed	Standard	1 GiB - 1 TiB	40-200

• Termination protection is turned off by default, you must turn it on.
• On an EBS-backed instance, the default action is for the root EBS volume to be deleted when the instance is terminated.
• EBS Root Volumes of your DEFAULT AMI’s CAN be encrypted. You can use a third party tool (such as bit locker etc) to encrypt the root volume. or this can be done when creating AMI’s in the AWS console or using the API.
• Additional volumes can be encrypted.
• As of Feb 2020 you can attach certain types of EBS volumes to multiple EC2 instances.

EBS Volume & Snapshot

• It’s really important to remember that your EBS volumes will always be in the same Availability Zones as your EC2 instance.
• Volumes exist on EBS. Think of EBS as a virtual hard disk.
• Snapshots exist on S3. Think of snapshots as a photograph of the disk.
• Snapshots are point in time copies of Volumes.
• Snapshots are incremental — this means that only the blocks that have changed since your last snapshot are moved to S3.
• If this is your first snapshot, it may take some time to create.
• To create a snapshot for Amazon EBS volumes that serve as root devices, you should stop the instance before taking the snapshot. However you can take a snap while the instance is running.
• You can create AMI’s from Snapshots.
• You can change EBS volume sizes on the fly, including changing the size and storage type.

Migrating EBS

• To move an EC2 volume from one AZ to another, take a snapshot of it, create an AMI from the snapshot and then use the AMI to launch the EC2 instance in a new AZ.
• To move an EC2 volume from one region to another, take a snapshot of it, create an AMI from the snapshot and the copy the AMI from one region to the other. Then use the copied AMI to launch the new EC2 instance in the new region.

AMI Type (EBS vs Instance Store)

You can select your AMI based on:

• Region (see Regions and Available Zones)
• Operation system
• Architecture (32-bit or 64-bit)
• Launch permissions
• Storage for the Root Device (Root Device Volume)
  • Instance Store (EPHEMERAL STORAGE)
  • EBS Backed Volumes

All AMIs are categorized as either backed by Amazon EBS or backed by instance store.

• For EBS Volumes: The root device for an instance launched from the AMI is an Amazon EBS volume created from an Amazon EBS snapshot.
• For Instance Store Volumes: The root device for an instance launched from the AMI is an instance store volume created from a template stored in Amazon S3.

Exam Tips

• Instance Store Volumes are sometimes called Ephemeral Storage.(For some reason they’re stopped, you are going to lose all of your data instant.)
• Instance store volumes cannot be stopped. If the underlying host fails, you will lose your data.
• EBS backed instances can be stopped. You will not lose the data on this instance if it is stopped.
• You can reboot both, you will not lose your data.
• By default, both ROOT volumes will be deleted on termination. However, with EBS volumes, you can tell AWS to keep the root device volume.

ENI vs ENA vs EFA

Terminology

• ENI

  For basic networking.

  • Elastic Network Interface - essentially a virtual network card.
  • An ENI is simply a virtual network card for your EC2 instances.

• EN

  For when you need speeds between 10 Gbps and 100 Gbps. Anywhere you need reliable, high throughput.

  • Enhanced Networking. Uses single root I/O virtualization (SR-IOV) to provide high-performance networking capabilities on supported instance types.
  • Enhanced networking provides higher bandwidth, higher packet per second (PPS) performance, and consistently lower inter-instance latencies. There is no additional charge for using enhanced networking.
  • Use where you want good network performance.
  • Depending on your instance type, enhanced networking can be enabled using:
    • Elastic Network Adapter(ENA), which supports network speeds of up to 100 Gbps for supported instance types.
    • Intel 82599 Virtual Function (VF) interface, which supports network speeds of up to 10 Gbps for supported instance types. This is typically used on older instances.
    • In any scenario question, you probably want to choose ENA over VF if given the option.

• EFA

  Fort when you need to accelerate High Performance Computing (HPC) and machine learning applications or if you need to do an OS-bypass.

  • Elastic Fabric Adapter (EFA) is a network device that you can attach to your Amazon EC2 instance to accelerate High Performance Computing (HPC) and machine learning applications.
  • EFA provided lower and more consistent latency and higher throughput than the TCP transport traditionally used in cloud-based HPC systems.
  • EFA can use OS-bypass. OS-bypass enables HPC and machine learning applications to bypass the operating system kernel and to communicate directly with the EFA device. It makes it a lot faster with a lot lower latency. Not supported with Windows currently, only Linux.

Encrypted Root Device Volumes & Snapshots

EBS Encryption

• snapshots of encrypted volumes are encrypted automatically.
• Volumes restored from encrypted snapshots are encrypted automatically.
• You can share snapshots, but only if they are unencrypted.
• These snapshots can be shared with other AWS accounts or made public.

You can now encrypt root device volumes upon creation of the EC2 instance. The process for making it encrypted:

• Create a Snapshot of the unencrypted root device volume
• Create a copy of the Snapshot and select the encrypt option
• Create an AMI from the encrypted Snapshot
• Use that AMI to launch new encrypted instance

Spot Instances & Spot Fleets

Amazon EC2 Spot Instances let you take advantage of unused EC2 capacity in the AWS Cloud.

• You may also use a Spot block to stop your Spot Instances from being terminated even if the Spot price goes over your max Spot price.

Spot Fleets

A spot Fleet is a collection of Spot Instances and, optionally, On-Demand Instances.

• The Spot Fleet attempts to launch the number of Spot Instances and On-Demand Instances to meet the target capacity you specified in the Spot Fleet request. The request for Spot Instances is fulfilled if there is available capacity and the maximum price you specified in the request exceeds the current Spot price. The spot Fleet also attempts to maintain its target capacity fleet if your Spot Instances are interrupted.

Exam Tips

• Spot Instances save up to 90% of the cost On-Demand Instances.
• Useful for any type of computing where you don’t need persistent storage.
• You can block Spot Instances from terminating by using Spot block.
• A Spot Fleet is a collection of Spot Instances and optionally, On-Demand Instances.

EBS Behaviors Reviewed

If we stop the instance, the data is kept on the disk (with EBS) and will remain on the disk until the EC2 instance is started.

If the instance is terminated, then by default the root device volume will also be terminated.

EC2 Hibernate Reviewed

When you hibernate an EC2 instance, the operating system is told to perform hibernation (suspend-to-disk). Hibernation saves the contents from the instance memory (RAM) to your Amazon EBS root volume. We persist the instance’s Amazon EBS root volume and any attached Amazon EBS data volumes.

When you start your instance out of hibernation:

• The Amazon EBS root volume is restored to its previous sate
• The RAM contents are reloaded
• The processes that were preciously running on the instance are resumed
• Previously attached data volumes are reattached and the instance retains its instance ID.

This is useful for:

• Long-running processes
• Services that take time to initialize

Exam Tips:

• EC2 Hibernate preserves the in-memory RAM on persistent storage(EBS)
• Much faster to boot up because you do not need to reload the operating system.
• Instance RAM must be less than 150 GB.
• Available for Windows, Amazon Linux 2 AMI, and Ubuntu.
• Instances can’t be hibernated for more than 60 days.

Cloud Watch

Amazon CloudWatch is a monitoring service to monitor your AWS resources, as well as the applications that you run on AWS.

CloudWatch can monitor things like:

• Compute
• Storage & Content Delivery

Host Level Metrics Consist of:

• CPU
• Network
• Disk
• Status Check

What Can I do with CloudWatch?

• Dashboards - Creates awesome dashboards to see what is happening with your AWS environments.
• Alarms - Allows you to set Alarms that notify you when particular thresholds are hit.
• Events - CloudWatch Events helps you to respond to state changes in your AWS resources.
• Logs - CloudWatch Logs helps you to aggregate, monitor, and store logs.

AWS CloudTrail

AWS CloudTrail increases visibility into your user and resource activity by recording AWS Management Console actions and API calls. You can identity which users and accounts called AWS, the source IP address from which the calls were made, and when the calls occurred.

CloudTrail vs CloudWatch

• CloudWatch monitors performance.
• CloudTrail monitors API calls in the AWS platform

Things need to be remembered:

• CloudWatch is used for monitoring performance.
• CloudWatch can monitor most of AWS as well as your applications that run on AWS.
• CloudWatch with EC2 will monitor events every 5 minutes by default.
• You can have 1 minute intervals by turning on detailed monitoring.
• You can create CloudWatch alarms which trigger notifications.
• CloudWatch is all about performance. CloudTrail is all about auditing.

Exam Tips

• Standard Monitoring = 5 Minutes
• Detailed Monitoring = 1 Minutes

AWS Command Line

• You can interact with AWS from anywhere in the world just by using the command line (CLI).
• You will need to set up access in IAM.
• Commands themselves are not in the exam, but some basic commands will be useful to know for real life.

Role: Identity Access Management Roles

• Roles are far more secure than storing your access key and secret access key on individual EC2 instances.
• Roles are easier to manage.
• Roles can be assigned to an EC2 instance after it is created using both the console & command line.
• Roles are universal — you can use them in any region.

BootStrap Scrips

Bootstrap Scripts are super powerful as you can see it’s a way of automating your infrastructure.

• Bootstrap scripts run when an EC2 instance first boots.

Instance Meta data

• Used to get information about an instance (such as public IP)

  curl http://196.254.169.254/latest/meta-data/

• User data simply contain the bootstrap script that you run

  curl http://196.254.169.254/latest/user-data/

EFS (Amazon Elastic File System)

Amazon Elastic File System(Amazon EFS) is as file storage service for Amazon Elastic Compute Cloud (EC2) instance. Amazon EFS is easy to use and provides a simple interface that allows you to create and configure file systems quickly and easily.

With Amazon EFS, storage capacity is elastic, growing and shrinking automatically as you add and remove files, so you applications have the storage they need, when they need it.

Exam Tips

• Supports the Network File System version 4 (NFSv4) protocol.
• You only pay for the storage you use (no pre-provisioning required)
• Can scale up to the petabytes
• Can support thousands of concurrent NFS connections
• Data is stored across multiple AZ’s within a region
• Read After Write Consistency

FSX for Windows & FSX for Lustre

Amazon FSx for Windows

• Amazon FSx for Windows File Server provides a fully managed native Microsoft Windows file system so you can easily move your Windows-based applications that require file storage to AWS. Amazon FSx is built on Windows Server.

FSX for Lustre

• Amazon FSx for Lustre is a fully managed file system that is optimized for compute-intensive workloads, such as high-performance computing, machine learning, media data processing, and electronic design automation (EDA).

• With Amazon FSx, you can launch and run a Lustre file system that can process massive data sets at up to hundreds of gigabytes per second of throughput, millions of IOPS, and sub-millisecond latencies.

How is FSx (Windows & Lustre) Different to EFS

Windows FSx	Lustre FSx	EFS
A managed Windows Server	A managed file system	A managed NAS filter for EC2 instances
Running Windows Server Message Block (SMB)-based file services		Based on Network File System (NFS) version 4
Designed for Windows and Windows applications	Designed specifically for fast processing of workloads	One of the first network file sharing protocols native to Unix and Linux
When you need centralized Storage for Windows-based applications	When you need high-speed, high-capacity distributed storage.	When you need distributed, highly resilient storage for Linux-based applications

EC2 Placement Groups

Clustered Placement Group

• Grouping of instances within a single Availability Zone.
• Recommended for applications that need low network latency, high network throughput, or both.
• Only certain instances can be launched in to a Clustered Placement Group (Compute Optimized, GPU, Memory Optimized, Storage Optimized).
• AWS recommend homogeneous instances within clustered placement groups.

Spread placement Group

• Group of instances that are each placed on distinct underlying hardware.
• Recommended for applications that have a small number of critical instances that should be kept separate from each other.
• Spread placement groups have a specific limitation that you can only have a maximum of 7 running instances per Availability Zone.

Partitioned

• Amazon EC2 divides each group into logical segments called partitions.
• Amazon EC2 ensures that each partition within a placement group have its own set of racks.
• Uses cases are multiple EC2 instances HDFS, Hbase, and Cassandra.
  • HDFS: The Hadoop Distributed File System ( HDFS ) is a distributed file system designed to run on commodity hardware.
  • Hbase: HBase is an open-source non-relational distributed database modeled after Google’s Bigtable and written in Java). It is developed as part of Apache Software Foundation‘s Apache Hadoop project and runs on top of HDFS (Hadoop Distributed File System) or Alluxio, providing Bigtable-like capabilities for Hadoop.
  • Cassandra: Apache Cassandra™ is a distributed NoSQL database that delivers continuous availability, high performance, and linear scalability that successful applications require.

Other Exam Tips

• The name you specify for a placement group must be unique within your AWS account.
• You can’t merge placement groups.
• You can’t move an existing instance into a placement group. Before you move the instance, the instance must be in the stopped state. You can move or remove an instance using the AWS CLI or an AWS SDK (software development kit), you can’t do it via the console yet.

HPC on AWS

You can create a large number of resources in almost no time. You only pay for the resources you use — and, once finished, you can destroy the resources.

• Data Transfer
  • Snowball, snowmobile (terabytes/petabytes worth of data)
  • AWS DataSync
  • Direct Connect (a cloud service solution that makes it easy to establish a dedicated network connection from your premises to AWS)
• Compute
  • EC2
  • EC2 fleets (Spot Instances or Spot Fleets)
  • Placement groups (cluster placement groups)
• Networking
  • Enhanced Networking (Uses single root I/O virtualization (SR-IOV) to provide high-performance networking capabilities on supported instance types. It provides higher I/O performance and lower CPU utilization)
  • Elastic Network Adapter
  • Elastic Fabric Adapters (Network device you can attach to your Amazon EC2 to accelerate HPC and machine learning applications.)
• Storage
  • Instance-attached storage
    • EBS(Amazon Elastic Block Store provides persistent block storage volumes for use with Amazon EC2 instances in the AWS Cloud.)
    • Instance Store
  • Network Storage
    • Amazon S3
    • Amazon EFS (Amazon Elastic File System is as file storage service for Amazon Elastic Compute Cloud (EC2) instance)
    • Amazon FSx for Lustre (Amazon FSx for Lustre is a fully managed file system that is optimized for compute-intensive workloads)
• Orchestration & Automation
  • AWS Batch
    • AWS Batch enables developers, scientist, and engineers to easily and efficiently run hundreds of thousands of batch computing jobs on AWS.
    • AWS Batch supports multi-node parallel jobs, which allows you to run a single job that spans multiple EC2 instance.
    • You can easily schedule jobs and launch EC2 instances according to your needs.
  • AWS ParallelCluster
    • Open-source cluster management tool that makes it easy for you to deploy and manage HPC clusters on AWS.
    • ParallelCluster uses a simple text file to model and provision all the resources needed for your HPC applications in an automated and secure manner.
    • Automate creation of VPC, subnet, cluster type, and instance types.

AWS WAF (Web Application Firewall)

AWS WAF is a web application firewall that lets you monitor the HTTP and HTTPS requests that are forwarded to Amazon CloudFront, an Application Load Balancer or API Gateway.

AWS WAF also lets you control access to your content.

• You can configure conditions such as what IP addresses are allowed to make this request or what query string parameters need to be passed for the request to be allowed.
• Then the application load balancer or CloudFront or API Gateway will either allow this content to be received or to give a HTTP 403 Status Code.

How to block malicious IP addresses:

• Use AWS WAF
• Use Network ACLs

Other Exam Tips

Get Hands Dirty

• Termination Protection is turned off by default, you must turn it on.
• On an EBS-backed instance, the default action is for the root EBS volume to be deleted when the instance is terminated. But any additional volumes by default won’t be deleted.
• EBS Root Volumes of your DEFAULT AMI’s CAN be encrypted. You can also use a third party tool (such as bit locker etc) to encrypt the root volume, or this can be done when creating AMI’s in the AWS console or using the API.
• Additional volumes can be encrypted.

---

REMEMBER TO READ FAQ

https://aws.amazon.com/ec2/faqs/