AWS Lambda

AWS Lambda is a service that provides you the option to run function written in .net core, node, python, java, etc to run on aws. AWS Lambda executes code only when needed and scales automatically. You pay only for the compute time you consume and no code when your code is not running, which is different than code running in Ec2 instance or ECS container. 

When a function is invoked, AWS lambda creates an instance and runs its handler method. While the first event is being processed, if the function is invoked again, and no instance is available, AWS lambda will create another instance. After processing the event the instance sticks around to process additional events. When a new instance is created, the response time increases which is called cold start. As more events come in, Lambda routes them to available instances and creates new instances as needed. Your function's concurrency is the number of instances serving requests at a given time. When the number of requests decreases, Lambda stops unused instances to free upscaling capacity for other functions. There is a limit to concurrency limit based on the region. When requests come in faster than your function can scale, or when your function is at maximum concurrency, additional requests fail with a throttling error.

To ensure that a function can always reach a certain level of concurrency, as a maximum allowed concurrency is shared across all functions in an account, you can configure the function with reserved concurrency. When a function has reserved concurrency, no other function can use that concurrency. Reserved concurrency also limits the maximum concurrency for the function.

To enable your function to scale without fluctuations in latency, you can use provisioned concurrency. It also integrates with auto-scaling, so you can update provisioned concurrency based on the demand.  The overhead in starting lambda (cold start) consists of two parts - time to set up an execution environment that is entirely controlled by AWS, and code initialization which involves things like initializing objects and frameworks. Provisioned Concurrency targets both causes of cold-start latency

You configure the amount of memory which will be available during execution. Similarly, you configure timeout which is the maximum time a function can run.  All AWS Lambda functions run securely inside a default system-managed Virtual Private Cloud (VPC) however you can configure it to run within custom VPC, subnet (use at least two subnets for high availability), and Security Group. When you enable a VPC, your Lambda function loses default internet access. If you require external internet access for your function, make sure that your security group allows outbound connections and that your VPC has a NAT gateway.

Lambda function and trigger (like API gateway) are the core component of AWS lambda. You specify the execution role is the IAM role that AWS Lambda assumes when it executes your function.

Handy Cli Command
aws lambda help
aws lambda update-function-code --profile <profile-name-if-not-default> --function-name  <function-name> --zip-file fileb://lambda-build/deploy-package.zip

AWS ECS

ECS is AWS’s proprietary, managed container orchestration service that supports Docker, much like google open-sourced container orchestration platform Kubernetes. Amazon also supports Kubernetes with their EKS offering more on that later. Running applications in containers rather than traditional VM’s brings great value due to the fact that they are easily scalable and ephemeral. However operating at scale, a container orchestration platform that automates the provisioning and deployment, redundancy and availability, scaling up\down based on a given load, resource allocation, health monitoring of containers and hosts, seamless deployment of new application versions, has become necessary. There are two ways to launch containers for both ECS and EKS and a factor to choose between them depends on how much control you want to have at the cluster host.

EC2 - You are responsible for deploying and managing your own cluster of EC2 instances for running the containers. Here the billing is based on the cost of the underlying EC2 instances and it's your responsibility to make sure your containers are densely packed and also your instance has all updates. This is more suitable for a large workload which requires more CPU and memory (or any special requirement) where you can optimize pricing by taking advantage of using spot instance or reserved instance.

AWS Fargate - You run your containers directly, without any EC2 instances. Here the billing is based CPU cores, and memory your task requires, per second. This is more suitable for small workloads with occasional bursts and also for the case where you do not want to manage the overhead of underlaying host.

ECS Cluster

Its a logical grouping of task or service. There are two ways to create a cluster: Network only and Ec2 with Networking. In the case of Ec2 launch type its a group of the container instances. You can assume this as a group of container instances (in case of Ec2 launch type you manage and create those instances, in case of Fargate, Amazon does that for you) acting as a single resource. You can mix and match instance types within a cluster but instances cannot join multiple clusters. A task is scheduled onto clusters. Handy aws cli command around ecs cluster

aws ecs create-cluster --cluster-name mycluster

aws ecs list-clusters

aws ecs describe-clusters --cluster mycluster

aws ecs delete-cluster --cluster mycluster

Container Agent

It allows container instances to join a cluster and it runs on Ec2

Container Instances

Ec2 instance registers to a cluster and connects via a container agent. The state can be active and connected, active and disconnect (in case if the instance is stopped) and finally inactive (terminated instance)

Task Definition

This describes how the docker image should run. This can include one or more containers. Grouped containers run in the same instance. This will help the interaction between containers in terms of latency. There are three main components of task definition

Family

Following are few of the important configuration which belongs to this

• Task Execution Role - This role is required by tasks to pull container images and publish container logs to Amazon CloudWatch on your behalf. 
• Task Role - IAM role that tasks can use to make API requests to authorized AWS services. For example, you can have a role configured to access S3 bucket and the task running can access s3 bucket.
• Network Mode - awsvpc, bridge, host, none

Container Definition 

This includes the following 

• image 
• port mapping - You cannot run two container instances on the same ec2 instance which uses the  same port. Container ports and protocols combination must be unique within a Task definition
• environment variables and entry point, 
• CPU - CPU unit reserved for the container, maps to CpuShares in docker run
• Memory - Amount of memory for a container, the sum of all container memory within a task should be less than task memory. The container will die if it exceeds the memory limit. This maps to memory docker run
• Memory reservation - When system memory is under heavy contention, Docker attempts to keep the container memory to this soft limit. Hard limit is the maximum it can use after which container will die. This maps to MemoryReservation in docker run.
• Logging configuration - You can configure your container instances to send log information to cloud watch or splunk etc 

Volumes

This is used for persisting data generated by and used by Docker containers.

You can define multiple containers in a task definition, though you should be careful in creating task definitions that group the containers that are used for a common purpose, and separate the different components into multiple task definition

Task Networking in AWS Fargate Task Definition

awsvpc network mode gives Amazon ECS tasks ENI, private IP and also provides greater security for your containers by allowing you to use security groups and network monitoring tools ( like VPC Flow Logs ) at a more granular level within your tasks. Containers belonging to the same task can communicate over the localhost interface.  When running a task in Fargate, there are two different forms of networking to consider:

Container (local) networking - Container networking is often used for tightly coupled application components, so it bypasses the network interface hardware and instead of the operating system just routes network calls from one process to the other directly and hence results in faster communication. Fargate uses a special container networking mode called awsvpc, which gives all the containers in a task a shared elastic network interface to use for communication. If you specify a port mapping for each container in the task, then the containers can communicate with each other on that port like 127.0.0.1:8080. However, when you deploy one or more containers as part of the same task they are always deployed together so it removes the ability to independently scale different types of workload up and down.

External networking - External networking is used for network communications that go outside the task to other servers that are not part of the task, or network communications that originate from other hosts on the internet and are directed to the task.

Scheduler

Scheduler helps you utilize your cluster resources. You don't have to figure out which Ec2 instance in the cluster will be running the task unless you have a specific need. There are three ways to schedule something on your cluster

• Services - These are long-lived and stateless. You define how many task instance will be running. It plays nicely with LB which load balances traffic to multiple task instances. Three steps of running task into a cluster
• Based on task definition it will figure out which container instances are available to run the task
• Figure out which AZ has the least amount of service task running
• Figure out which instance has the least amount of task running

• Task - These are sort lived/1 off the task that exit when done. You can use run task command which distributes tasks on your cluster and minimizes specific instances from getting overloaded.
• Starting Task - StartTask will let you pick where you want to run the task. It will let you build or use your own scheduler.

Both service and task have three states - pending, running, stopped. The container agent is responsible for state tracking.

AWS ECS Autoscaling

AWS Auto Scaling automates capacity management for various AWS services. For ECS you specify the minimum/maximum number of the task which sets the boundary of the autoscaling policy. You also need to specify the IAM role that authorizes ECS to use autoscaling service. You can create an IAM role for service application-autoscaling.amazonaws.com and attach policy AmazonEC2ContainerServiceAutoscaleRole to it.

Autoscaling Policy

You can define target tracking or step-scaling policy. For target tracking, you can specify a target value for Average CPU Utilization, Average Memory Utilization, or Request Count, based on which, your service will scale out or scale in. For step-scaling, you can use cloud watch alarm, with a threshold, based on which you can add or remove the task. Cloud watch alarm for ECS fragate, supports CPU Utilization and Memory Utilization ECS service matrix. It also supports the cooldown period, which is the duration for which the next scaling operation is hold off.

Handly aws cli command around ecs service/tasks

aws ecs create-service --generate-cli-skeleton

aws ecs create-service --cluster mycluster --service-name web --task-definition web --desired-count 1

aws ecs list-services --cluster mycluster

aws ecs describe-services --cluster mycluster --services web

aws ecs update-service --cluster mycluster --service-name web --task-definition web --desired-count 2

aws ecs delete-service --cluster mycluster --service-name web 

aws ecs register-task-definition --generate-cli-skeleton

aws ecs run-task --cluster mycluster --task-definition web --count 1

aws ecs list-tasks --cluster mycluster

aws ecs stop-task --cluster mycluster --task arn

aws ecs list-container-instances --cluster mycluster

aws ecs start-task --cluster mycluster --task-definition web --container-instances arn

How to deal with APIs breaking change

Microservices brings down a big challenge around breaking contracts. This can impact the existing consumer, so my take has always been to have all minor changes in APIs to be backward compatible, whether it is exposed to internal or external clients. In case your change is big enough, then you may need to version your API. 

To have some kind of check around this, you can have a unit test suite that does the API integration test. Though this is easy, the challenge here is that the developer can fix it as per the new API definition and miss how a consumer might be using it. The next option is an automated contract testing (tools like PACT). This ensures that a pair of applications work correctly together as per the contracted document. This is definitely a great tool but on the other side, it does require effort in writing it and maintaining it. In my opinion, this is more suitable for application which has a high level of DevOps maturity. The next option is versioning, maybe the safest bet as it has very minimal risk and rolling back is also straight forward, but on the other side maintaining versioning for every minor change will very easily result in the messy application if it's frequently updated. The next and the final one which I am going to talk here is tools like swagger-diff. This is a simple tool that requires minimal effort to integrate it with your pipeline. The goal of this tool is to warn you if you have a breaking change. You can use the node module in the following way which will give you the differences

swagger-diff <old> <new>

Next, I am going to talk about types of breaking change and how to deal with that

Updating resource path

This can be done smoothly by providing support to both the old resource path and the updated resource path. The old resource path can redirect to the new resource path. In asp.net core you can use RedirectToActionPermanent which will result in 301 and the user will be redirected to the new action.

Adding Required Parameter 

Here you need to provide support to the resource with both old parameters and with the new parameter. You can do this by having two model objects one with old parameters for the old resource (which will make sure the existing contract is not breaking) and the other with new parameters. You can also mark the old one as obsolete is you don't plan to support it in the future.

The same can be a case when you add optional parameters that change the default behavior of invocation.

Changing Response Structure Or Response Code

We often consider additive changes are non-breaking as the consumer can safely ignore it until they upgrade their application, which is for most cases true, but there are cases where additive change can be breaking also like database truncation, etc.

There are other scenarios like change is rate-limiting rules, caching mechanism may also sometime be a breaking change.

Having to support multiple implementations also becomes challenging, so it's good to come up with a plan (with notice to consumers) to remove obsolete resources. 

Terraform

Terraform is an open-source infrastructure as a code (hashicorp configuration language) software tool created by HashiCorp. Similar to CloudFormation, terraform is designed as a provisioning tool, which is different than tools like chef and puppet, which are primarily designed for configuration management. Definitely, there is some overlap between configuration management tools and provisioning tools, but it's important to understand what they are best at. Terraform and CloudFormation follows declarative state language, which means you specify end state and tool will figure out sequence or dependencies of the task. Tools like ansible are procedural-based automation tools, where you write code and specify the sequence of steps to achieve end state. 

The main purpose of the Terraform language is declaring resources. All other feature exists to make the definition of resources more flexible and convenient. A Terraform configuration consists of a root module, where evaluation begins, along with a tree of child modules created when one module calls another.

Terraform Components

variable

Input variables serve as parameters for a Terraform module. When you declare variables in the root module of your configuration, you can set their values using CLI options (-var="my_var=something"), or in .tfvars file or in environment variables. When you declare them in child modules, the calling module should pass values in the module block. The value assigned to a variable can be accessed (var.my_var) only from expressions within the module where it was declared. Ex string, number, bool, list, map, set

provider

Provider (like aws or google) requires configuration of their own, like authentication settings, etc. You can have multiple providers give them an alias and reference it by alias name (<PROVIDER NAME>.<ALIAS>). Every time a new provider is added, you need to download it by initializing it.

resource

Resource block describes one or more infrastructure objects. Each resource is associated with a single resource type. Terraform handles most resource dependencies automatically but in the rare cases where you need to explicitly define you can use depends_on metadata argument. You can use a count meta argument to create multiple resources. You can add Provisioners to any resource and are used to execute scripts on a local or remote machine

output

This is like the return of a module that can be used in the child module or for printing from the root module.

module 

This is a container for multiple resources while it helps reusability of the code. Every Terraform configuration has at least a root module, which consists of the resources defined in the .tf files in the main working directory.

data

Data sources allow fetching data defined outside of Terraform. One of the very common use case of this is getting aws AZ

data "aws_availability_zones" "available" {
  state = "available"
}

In the above example, we are storing aws AZs list which can be accessed anywhere in the code as follow

"${data.aws_availability_zones.available.names[0]}"

State

Terraform state can be considered some sort of database to map Terrform config to the real world. This also tracks resource dependencies.  This can be saved locally or remotely and location can be configured by -state. It locks state while applying for all operations that could write state.

Terraform Console

This provides an interactive way to evaluate an expression. This becomes very handy to test with build-in function and also with interpolation. Just type terraform console on the command prompt and then play with it.

ES6 New Fetures

let and const

Hoisting is JavaScript's default behavior of moving all declarations to the top of the current scope, which means following is valid

 x=5
 var x
 y === 7 //false, as declaration is hoisted but not the initializations
 var y = 7;

JavaScript only hoists declarations, not initializations.

var is function scoped which sometimes is confusing. Refer following example

 if(true){
   var x = 20
 }
 console.log(x) //this will log 20

Even though x is declared inside if block, but since var declaration are hoisted it will be accessible anywhere within the function.

In ES6, let and const was introduced which is blocked scoped and is align with other programmable language like c#, java. In the above example, if you change var to let, it will throw error as you try to access x outside if block where its declared.

Arrow function

This is defined as anonymous function much like lambda function in c#

Destructuring Assignment

This make it possible to unpack values from array, or properties from objects, into distinct variables.

[a, b, ...rest] = [10, 20, 30, 40, 50];//a=10,b=20,rest=[30,40,50]

const o = {p: 42, q: true};
const {p: foo, q: bar} = o;

function someMethod({p: foo, q: bar})

This can be used even for nested object.

Rest parameter give you access to remaining items in the from of array which makes it lot easier to run any of array method on that.

Spread syntax allows an iterable such as an array expression or string to be expanded in places where zero or more arguments (for function calls) or elements (for array literals) are expected, or an object expression to be expanded in places where zero or more key-value pairs (for object literals) are expected
var arr2 = [...arr]; // like arr.slice()
var concat = [...arr1, ...arr2];

var clonedObj = { ...obj1 };
var mergedObj = { ...obj1, ...obj2 };

Template literals

`Hello ${name} !!`

Can also be used with multi line

ES6 Module

ES6 provides build-in module in javascript, earlier to this you have to use library like commonjs, amd

ES6 Class

.net garbage collection

The garbage collector serves as an automatic memory manager. After the garbage collector is initialized by the CLR, it allocates a segment of memory to store and manage objects. When a garbage collection is triggered, the garbage collector reclaims the memory (managed memory) that is occupied by dead objects. The reclaiming process compacts live objects so that they are moved together, and the dead space is removed, thereby making the heap smaller.

If your managed objects reference unmanaged objects, you will have to explicitly free them (unmanaged objects). .NET does not allocate Unmanaged memory, as it's coming from outside sources and thus the GC doesn't know about it.  For example, when you open a file (FileStream) you are basically calling (behind the scenes) the CreateFile unmanaged Win32 function. This function allocates an unmanaged file handle directly from the file system. .NET and the GC has strictly no way of tracking this unmanaged object and everything it does.

To handle this you have IDisposable and Dispose, which you will implement in your managed object, where you will clean up your unmanaged memory (if any created by your object) and implement the finalizer to call Dispose(). The finalizer is just a safeguard, it will make sure that those resources get cleaned up eventually if the caller forgets to dispose of your class properly.

.Net provides using statement, which is a convenient syntax that ensures that Dispose is called (even if an exception occurs in using block) as soon as execution reaches close brace.

Garbage collection occurs when the system has low physical memory or the memory used by allocated objects on the managed heap surpasses an acceptable threshold. You also have an option of calling GC.Collect, but most likely you do not have to call this, it's only in some unique situation you will be using this function.

The heap is organized into generations so it can handle long-lived or short-lived object. Refer following for more detail.