AWS Lambda

AWS Lambda is a service that provides you the option to run function written in .net core, node, python, java, etc to run on aws. AWS Lambda executes code only when needed and scales automatically. You pay only for the compute time you consume and no code when your code is not running, which is different than code running in Ec2 instance or ECS container. 

When a function is invoked, AWS lambda creates an instance and runs its handler method. While the first event is being processed, if the function is invoked again, and no instance is available, AWS lambda will create another instance. After processing the event the instance sticks around to process additional events. When a new instance is created, the response time increases which is called cold start. As more events come in, Lambda routes them to available instances and creates new instances as needed. Your function's concurrency is the number of instances serving requests at a given time. When the number of requests decreases, Lambda stops unused instances to free upscaling capacity for other functions. There is a limit to concurrency limit based on the region. When requests come in faster than your function can scale, or when your function is at maximum concurrency, additional requests fail with a throttling error.

To ensure that a function can always reach a certain level of concurrency, as a maximum allowed concurrency is shared across all functions in an account, you can configure the function with reserved concurrency. When a function has reserved concurrency, no other function can use that concurrency. Reserved concurrency also limits the maximum concurrency for the function.

To enable your function to scale without fluctuations in latency, you can use provisioned concurrency. It also integrates with auto-scaling, so you can update provisioned concurrency based on the demand.  The overhead in starting lambda (cold start) consists of two parts - time to set up an execution environment that is entirely controlled by AWS, and code initialization which involves things like initializing objects and frameworks. Provisioned Concurrency targets both causes of cold-start latency

You configure the amount of memory which will be available during execution. Similarly, you configure timeout which is the maximum time a function can run.  All AWS Lambda functions run securely inside a default system-managed Virtual Private Cloud (VPC) however you can configure it to run within custom VPC, subnet (use at least two subnets for high availability), and Security Group. When you enable a VPC, your Lambda function loses default internet access. If you require external internet access for your function, make sure that your security group allows outbound connections and that your VPC has a NAT gateway.

Lambda function and trigger (like API gateway) are the core component of AWS lambda. You specify the execution role is the IAM role that AWS Lambda assumes when it executes your function.

Handy Cli Command
aws lambda help
aws lambda update-function-code --profile <profile-name-if-not-default> --function-name  <function-name> --zip-file fileb://lambda-build/deploy-package.zip

AWS ECS

ECS is AWS’s proprietary, managed container orchestration service that supports Docker, much like google open-sourced container orchestration platform Kubernetes. Amazon also supports Kubernetes with their EKS offering more on that later. Running applications in containers rather than traditional VM’s brings great value due to the fact that they are easily scalable and ephemeral. However operating at scale, a container orchestration platform that automates the provisioning and deployment, redundancy and availability, scaling up\down based on a given load, resource allocation, health monitoring of containers and hosts, seamless deployment of new application versions, has become necessary. There are two ways to launch containers for both ECS and EKS and a factor to choose between them depends on how much control you want to have at the cluster host.

EC2 - You are responsible for deploying and managing your own cluster of EC2 instances for running the containers. Here the billing is based on the cost of the underlying EC2 instances and it's your responsibility to make sure your containers are densely packed and also your instance has all updates. This is more suitable for a large workload which requires more CPU and memory (or any special requirement) where you can optimize pricing by taking advantage of using spot instance or reserved instance.

AWS Fargate - You run your containers directly, without any EC2 instances. Here the billing is based CPU cores, and memory your task requires, per second. This is more suitable for small workloads with occasional bursts and also for the case where you do not want to manage the overhead of underlaying host.

ECS Cluster

Its a logical grouping of task or service. There are two ways to create a cluster: Network only and Ec2 with Networking. In the case of Ec2 launch type its a group of the container instances. You can assume this as a group of container instances (in case of Ec2 launch type you manage and create those instances, in case of Fargate, Amazon does that for you) acting as a single resource. You can mix and match instance types within a cluster but instances cannot join multiple clusters. A task is scheduled onto clusters. Handy aws cli command around ecs cluster

aws ecs create-cluster --cluster-name mycluster

aws ecs list-clusters

aws ecs describe-clusters --cluster mycluster

aws ecs delete-cluster --cluster mycluster

Container Agent

It allows container instances to join a cluster and it runs on Ec2

Container Instances

Ec2 instance registers to a cluster and connects via a container agent. The state can be active and connected, active and disconnect (in case if the instance is stopped) and finally inactive (terminated instance)

Task Definition

This describes how the docker image should run. This can include one or more containers. Grouped containers run in the same instance. This will help the interaction between containers in terms of latency. There are three main components of task definition

Family

Following are few of the important configuration which belongs to this

• Task Execution Role - This role is required by tasks to pull container images and publish container logs to Amazon CloudWatch on your behalf. 
• Task Role - IAM role that tasks can use to make API requests to authorized AWS services. For example, you can have a role configured to access S3 bucket and the task running can access s3 bucket.
• Network Mode - awsvpc, bridge, host, none

Container Definition 

This includes the following 

• image 
• port mapping - You cannot run two container instances on the same ec2 instance which uses the  same port. Container ports and protocols combination must be unique within a Task definition
• environment variables and entry point, 
• CPU - CPU unit reserved for the container, maps to CpuShares in docker run
• Memory - Amount of memory for a container, the sum of all container memory within a task should be less than task memory. The container will die if it exceeds the memory limit. This maps to memory docker run
• Memory reservation - When system memory is under heavy contention, Docker attempts to keep the container memory to this soft limit. Hard limit is the maximum it can use after which container will die. This maps to MemoryReservation in docker run.
• Logging configuration - You can configure your container instances to send log information to cloud watch or splunk etc 

Volumes

This is used for persisting data generated by and used by Docker containers.

You can define multiple containers in a task definition, though you should be careful in creating task definitions that group the containers that are used for a common purpose, and separate the different components into multiple task definition

Task Networking in AWS Fargate Task Definition

awsvpc network mode gives Amazon ECS tasks ENI, private IP and also provides greater security for your containers by allowing you to use security groups and network monitoring tools ( like VPC Flow Logs ) at a more granular level within your tasks. Containers belonging to the same task can communicate over the localhost interface.  When running a task in Fargate, there are two different forms of networking to consider:

Container (local) networking - Container networking is often used for tightly coupled application components, so it bypasses the network interface hardware and instead of the operating system just routes network calls from one process to the other directly and hence results in faster communication. Fargate uses a special container networking mode called awsvpc, which gives all the containers in a task a shared elastic network interface to use for communication. If you specify a port mapping for each container in the task, then the containers can communicate with each other on that port like 127.0.0.1:8080. However, when you deploy one or more containers as part of the same task they are always deployed together so it removes the ability to independently scale different types of workload up and down.

External networking - External networking is used for network communications that go outside the task to other servers that are not part of the task, or network communications that originate from other hosts on the internet and are directed to the task.

Scheduler

Scheduler helps you utilize your cluster resources. You don't have to figure out which Ec2 instance in the cluster will be running the task unless you have a specific need. There are three ways to schedule something on your cluster

• Services - These are long-lived and stateless. You define how many task instance will be running. It plays nicely with LB which load balances traffic to multiple task instances. Three steps of running task into a cluster
• Based on task definition it will figure out which container instances are available to run the task
• Figure out which AZ has the least amount of service task running
• Figure out which instance has the least amount of task running

• Task - These are sort lived/1 off the task that exit when done. You can use run task command which distributes tasks on your cluster and minimizes specific instances from getting overloaded.
• Starting Task - StartTask will let you pick where you want to run the task. It will let you build or use your own scheduler.

Both service and task have three states - pending, running, stopped. The container agent is responsible for state tracking.

AWS ECS Autoscaling

AWS Auto Scaling automates capacity management for various AWS services. For ECS you specify the minimum/maximum number of the task which sets the boundary of the autoscaling policy. You also need to specify the IAM role that authorizes ECS to use autoscaling service. You can create an IAM role for service application-autoscaling.amazonaws.com and attach policy AmazonEC2ContainerServiceAutoscaleRole to it.

Autoscaling Policy

You can define target tracking or step-scaling policy. For target tracking, you can specify a target value for Average CPU Utilization, Average Memory Utilization, or Request Count, based on which, your service will scale out or scale in. For step-scaling, you can use cloud watch alarm, with a threshold, based on which you can add or remove the task. Cloud watch alarm for ECS fragate, supports CPU Utilization and Memory Utilization ECS service matrix. It also supports the cooldown period, which is the duration for which the next scaling operation is hold off.

Handly aws cli command around ecs service/tasks

aws ecs create-service --generate-cli-skeleton

aws ecs create-service --cluster mycluster --service-name web --task-definition web --desired-count 1

aws ecs list-services --cluster mycluster

aws ecs describe-services --cluster mycluster --services web

aws ecs update-service --cluster mycluster --service-name web --task-definition web --desired-count 2

aws ecs delete-service --cluster mycluster --service-name web 

aws ecs register-task-definition --generate-cli-skeleton

aws ecs run-task --cluster mycluster --task-definition web --count 1

aws ecs list-tasks --cluster mycluster

aws ecs stop-task --cluster mycluster --task arn

aws ecs list-container-instances --cluster mycluster

aws ecs start-task --cluster mycluster --task-definition web --container-instances arn