Aws-lambda | Mark Prager

AWS Lambda Example

So, you have heard a lot of talk about AWS lamda's serverless solution. What's it all about ?

Well, as you know, working with AWS can rack up a hefty bill, especially if you are running thousands of servers. However, not all the servers need to run as servers. Perhaps you only want to run something now and again ?, or you want something to trigger after another event. You should really consider - does this warrant setting up another server - with all its Operating System, TCP stack and software - running 24x7.

This is where AWS lamdba comes in. It allows you to create a function that will run somewhere in the cloudspace at a fraction of the cost, and without needing to worry about all that other overhead of installing operating systems and other suppporting software just to get your function to work.

I don't claim to be a AWS lambda expert, and this page just explains how I set up an AWS lambda function.

One of the things that can 'get lost' in all of AWS creating and terminating servers are orphaned volumes - that is EC2 volumes that are created can suddenly get orphaned - i.e. not attached to anything. These generally tend to pile up and can cost you a couple of dollars each time.

So lets set up a function to review your whole system daily, and report on any orphaned volumes that exist. You can then decide whether to do something about them in your spare time - but at least you are on the ball and know they exist.

The function itself is plagiarized from two sites:

· https://mike.lapidak.is/thoughts/lambda-tracking-orphaned-ebs-volumes

· http://rathinavneet.blogspot.com/2016/09/list-all-instances-of-aws-account-using.html

Step 1 - Create your lambda function:

Go to the AWS Lambda page, and select the "Create a Lambda Function" button.

Next, you will be asked to select a blueprint. There are around 90+ blueprints to choose from. In this case, I chose the 'Author from scratch' blueprint.

Next you will be asked to put in some basic permissions of the lambda function. lets put in the following values:

Lets use:

· Name: findOrphanedVolumes

· Role: Create a custom role

· Role Name: lambdaOrphanedVolumesRole

· choose Policy Template: The new custom role, you just created.

You will be taken to a new IAM role page - to create the new role:

After you create the role, add the additional policies shown:

So in the end, you will have a role containing:

· Basic Edge Lambda permissions

· EC2 read only permissions ,

· and SNS permissions.

After you have created the role select Create Function.

Step 2 - Writing the actual code

Well done !! You have created your lambda function - admittedly, it does not do very much at this point, but its the starting point.

So now, you need to choose your programming language to use - Currently, AWS supports the following:

· C#

· Java 8

· Node.js 4.3

· Node.js 6.10

· Python 2.7

· Python 3.6

Its a good idea to keep an eye on this, as AWS will likely change their support from time to time, and add new languages, and perhaps remove older ones.

In our example, we will use Python 2.7 ( as its what I am most familiar with) - Note: as you change languages the function wrapper will change.,

I generally edit inline (on screen) as its easier to manage. If you are writing a more complex function it might be easier to write it offline in an editor and upload it.

The code, I will use is as follows:

#!/usr/bin/env python

import boto3
from datetime import *
from os import environ

today = datetime.now().date()

def publish_sns(sns_message, sns_arn):
"""
Publish message to the master SNS topic
"""

sns_client = boto3.client('sns', region_name='us-east-1')

print "Publishing message to SNS topic..."
print sns_message
sns_client.publish(TargetArn=sns_arn, Message=sns_message, Subject='Orphaned Volume Report: ' + str(today))
return

def lambda_handler(event, context):
#set the date to today for the snapshot

client = boto3.client('ec2')
regions = client.describe_regions()['Regions']

missingReport = "Unattached Volumes Report - Date:" + str(today) + "\n"

x=0
# Connect to EC2
for region in regions:
region_name=region['RegionName']
missingReport += "\nBelow are the volumes that exist but are not attached in " + region_name + ":\n"
ec2 = boto3.resource('ec2',region_name=region['RegionName'])

volumes = ec2.volumes.all()
for v in volumes:
if v.state != "in-use":
missingReport += str(v.id) + " - Size: " + str(v.size) + " - Created: " + str(v.create_time) + " State: " + str (v.state) + "\n"
x += 1

if x > 0:
## There are unattached volumes
publish_sns(missingReport, environ['SNSArn'])

return 'lambda success'

Lets review this code before we continue -

The first part describes which python functions we need to include - namely -

· boto3 to access AWS function

· datetime for date and time function

· os environ for other operating system environment functions.

The function "publish_sns" is used to publish an sns message - in this case an email of the report to the required destination.

The second function "lambda_handler" is the function that is called when the lambda is triggered ( more of this later).

Going thru this function shows what happens:

First we create a boto3 client, and then get a list of all the regions, and then start to build up the email report.

Next, we loop thru each region, adding a title to the message for each region, and then cycling thru all the volumes looking for any that are "not in-use' . For any unattached volume, we add details of that volume to the report.

Finally, at the end of the function, we check if there were any volumes in fact found, and if so publish this report to an email address that is supplied in an environment variable.

For now, save your work.

Step 3 - Getting your email in for the report

Great, your function is set, but there are a few more things to take care of before it is ready.

Firstly, you need to get your email in - remember ? as part of your publish_sns function. For this we need to go an create an SNS topic to use -

Go to the AWS SNS form, and create a new topic - lets fill it in as follows:

After it is created, select the ARN used, and you will find an empty subscription page. From there, add in a new subscription with your email address

Once you have done that, AWS will send you an email at the specified address, and you should confirm the subscription.

Now that you have your email in the system, lets copy that ARN for the topic - and place it in the lambda page:

Step 4 - Other things to consider

Before continuing, there are a few last items to consider -

· Do you want to tag your function (always a good idea to) ?

· Do you want to attach your lambda to a specific VPC

· Set the timeout and basic memory setting.

This last one is particularly important - if your lambda is lengthy and requires more memory and time to run, you should consider increasing these. (in our example I increased it to 1 minute).

Now save your work

Step 5 - Testing your lambda

Now, you are ready to test your lambda. Select the test function.

If this is your first time - you will be asked to create a test data set. For our example, we dont really need a test data set - so we are just using a default value as shown above.

Click the test run, and hopefully you will run to success. If not, take a look where you are failing - perhaps you role and policy permissions are not 100% correct, or maybe the lambda timed out too early. If these things happen ( and they do !!) -fix them before continuing.

Step 6 - Triggering your lambda

Fantastic - you have a fully functioning lambda. However, you don't really want to go into this page daily to run this function, you want it to operate automatically.

Go to the "triggers' tag in the lambda function, and select - create a new trigger:

In this case - we need to add in what the trigger will be - so we select "CloudWatch Events", give it a rule name and description, and enter a schedule expression based on a cron value.

Note the format of the schedule expression. After you press submit - you should see something along the lines:

If you look carefully, you can see that you can trigger the lambda on a number of scenarios - based on cloudwatch logs or an alexa event and so on.

Anyway, that's it !! - You should be ready to go - and your lambda function can now be run daily, and provide you an orphaned disk report.

Comments welcome at markpragerinfo@gmail.com