gkrizek / bash-lambda-layer

Run Bash scripts in AWS Lambda via Layers
MIT License
428 stars 89 forks source link
aws bash lambda lambda-layer serverless

Bash in AWS Lambda

NO LONGER SUPPORTED

This repository and layer is no longer receiving updates or support. I've been too busy to keep up with changes in the Lambda environment and this has fallen behind. Use at your own risk.


Run Bash in AWS Lambda via Layers. This Layer is 100% Bash and handles all communication with the Lambda Runtime API. This allows you to run full Bash scripts and commands inside of AWS Lambda. This Layer also includes common CLI tools used in Bash scripts.

See the How To section to understand how to use these layers. Also see the example-basic.sh file for an example of how to write a Bash script compatible with this Layer.

ARN

arn:aws:lambda:<region>:744348701589:layer:bash:8

How To

Getting Started

AWS Lambda Console

  1. Login to your AWS Account and go to the Lambda Console.
  2. Create a new function and give it a name and an IAM Role.
  3. For the "Runtime" selection, select Use custom runtime in function code or layer.
  4. In the "Designer" section of your function dashboard, select the Layers box.
  5. Scroll down to the "Referenced Layers" section and click Add a layer.
  6. Select the Provide a layer version ARN option, then copy/paste the Layer ARN for your region.
  7. Click the Add button.
  8. Click Save in the upper right.
  9. Upload your code and start using Bash in AWS Lambda!

AWS CLI

  1. Create a function that uses the provided runtime and the Layer ARN for your region.
$ aws lambda create-function \
    --function-name bashFunction \
    --role bashFunctionRole \
    --handler index.handler \
    --runtime provided \
    --layers arn:aws:lambda:<region>:744348701589:layer:bash:8 \
    --zip-file fileb://function.zip
  1. Start using Bash in AWS Lambda!

Updating Versions

AWS Lambda Console

  1. In the "Designer" section of your function dashboard, select the Layers box.
  2. Scroll down to the "Referenced Layers" section and click Add a layer.
  3. Select the Provide a layer version ARN option, then copy/paste the Layer ARN for your region.
  4. Click the Add button.
  5. Still under the "Referenced Layers" section, select the previous version and click Remove.
  6. Click Save in the upper right.

AWS CLI

  1. Update your function's configration and add the Layer ARN for your region.
$ aws lambda update-function-configuration \
    --function-name bashFunction \
    --layers arn:aws:lambda:<region>:744348701589:layer:bash:8

Writing Scripts

Like any other Lambda function code, your main script's name must match the first part of your handler. Inside your main script, you must define a function that matches the second part of the handler. You must have set -e be the first line inside your function. Putting #!/bin/bash at the top of your file is not necessary. So if your Lambda handler is index.handler, your file and contents should look like:

$ cat index.sh
handler () {
    set -e
    ...
}

The event data is sent to your function as the first parameter. To access it, you should use $1. So if you need the event data, you should set it to a variable. For example, EVENT_DATA=$1.

handler () {
    set -e
    EVENT_DATA=$1
}

All the pre-installed tools are already in your $PATH so you can use them as expected. Any command output is automatically sent to CloudWatch, just like normal Lambda functions.

handler () {
    set -e
    EVENT_DATA=$1
    aws s3 ls $(echo $EVENT_DATA | jq ."bucket")
}

If you need to send a response back, you should send the response to stderr. (see the Caveats section for an explanation) To send output to stderr you should use >&2. This will be picked up and returned from the Lambda function.

handler () {
    set -e
    EVENT_DATA=$1
    aws s3 ls $(echo $EVENT_DATA | jq ."bucket")
    echo "{\"success\": true}" >&2
}

Caveats

Bash behaves in ways unlike other programming languages. As such, there are some requirements on the user's end that must be done.

Notes

Building

To build a layer, simply run make build. This will create a zip archive of the layer in the export/ directory.

Publishing

To publish the layer to the public, simply run make publish. This will create a new version of the layer from the export/layer.zip file (create from the Build step) and give it a global read permission.

Adding New Executables

Some executables are able to run by themselves and some require additional dependencies that are present on the server. It's hard to cover here case here, but if the executable run by itself it can easily be added. If it has dependencies, you must explore what those dependencies are and how to add them to the layer as well.

You can either add the executable from an Amazon Linux AMI or from the lambci/lambda:build-python3.6 Docker image.

Disclaimer: I usually don't add in executables from pull requests for security reasons. If you would like to see an executable in this layer make an issue and I'll try to add it.

Included Executables

Already included in the Lambda environment:

If you would like to see more, please create an issue.

Shout-out to the LambCI team for their work on lambci/git-lambda-layer which some of the git and ssh build process was taken from.