Add AWS CDK examples for creating and using a layer from the Serverless Application Repository

[austoonz]

What were you initially searching for in the docs? Using the AWS CDK, I wanted to deploy a Lambda layer from the Powertools Serverless Application Repository ARN for re-use within my CDK application.

The Lambda Layer documentation does not include how to do this.

Is this related to an existing part of the documentation? Please share a link https://awslabs.github.io/aws-lambda-powertools-python/#lambda-layer

Describe how we could make it clearer A code snippet and details could be provided to make it easier for a customer to get up and running faster.

If you have a proposed update, please share it here

Using similar language to the existing documentation, but some rewording could also used to provide the overall detail with SAM and CDK examples below it.

If using the AWS CDK, you can create include this SAR App and lock to a specific semantic version. The Layer can be used in the same CDK application. Once deployed, it'll be available across the account this is deployed to.

insert a TypeScript CDK example

# Create the AWS Lambda Powertools for Python layer
powertools_arn = 'arn:aws:serverlessrepo:eu-west-1:057560766410:applications/aws-lambda-powertools-python-layer'
powertools_application = aws_sam.CfnApplication(self, 'AWSLambdaPowertoolsApplication',
                                                location={'applicationId': powertools_arn,
                                                          'semanticVersion': '1.12.0'})
powertools_resource = cdk.CfnResource(powertools_application, 'AWSLambdaPowertoolsResource',
                                      type='AWS::Serverless::Application',
                                      properties={'Location': {
                                          'ApplicationId': powertools_arn,
                                          'SemanticVersion': '1.12.0'
                                      }})
powertools_layer = aws_lambda.LayerVersion.from_layer_version_arn(
    self, 'AWSLambdaPowertoolsLayer', powertools_resource.get_att("Outputs.LayerVersionArn").to_string())

# Reference the Layer in a Lambda definition
my_function = aws_lambda.Function(
    self, "MyFunction",
    layers=[powertools_layer],
    runtime=aws_lambda.Runtime.PYTHON_3_8,
)

[michaelbrewer]

Possibly a Python CDK example :P

[austoonz]

The example I included was Python CDK. 😁

[ran-isenberg]

+1 for this, was on my todo list for a long time ;)

[ran-isenberg]

is there a way to always get the latest published version?

[am29d]

Hi Andrew,

Thanks for opening the issue and providing the sample code. I will find a good spot in the docs to add it.

[am29d]

is there a way to always get the latest published version?

sadly no, you would need to run CLI or some custom resource to fetch it and sort by version

[michaelbrewer]

is there a way to always get the latest published version?

sadly no, you would need to run CLI or some custom resource to fetch it and sort by version

For CDK, I like to load that kind of thing via configuration settings.

[ran-isenberg]

is there a way to always get the latest published version?

sadly no, you would need to run CLI or some custom resource to fetch it and sort by version

For CDK, I like to load that kind of thing via configuration settings.

I will need to write a function that takes the powertools version from my pipfile.lock file and set the layer version parameter. annoying, but will work.

[michaelbrewer]

@risenberg-cyberark for docs i would just use the cdk.json or runtime context to configure the version of powertools.

[michaelbrewer]

@risenberg-cyberark @austoonz here is a full working example using Poetry:

Directory structure like below, with cdk folder for the AWS CDK code and hello-world with the lambda code.

.
├── cdk
│   ├── Makefile
│   ├── README.md
│   ├── app.py
│   ├── cdk.json
│   ├── lambda_stack
│   │   ├── __init__.py
│   │   └── lambda_stack.py
│   ├── poetry.lock
│   └── pyproject.toml
└── hello-world
    └── app.py

Source for: cdk/pyproject.toml

[tool.poetry]
name = "cdk"
version = "0.1.0"
description = "CDK code for the hello world example"
authors = ["Michael Brewer <michael.brewer@gyft.com>"]

[tool.poetry.dependencies]
python = "^3.8"
"aws-cdk.core" = "^1.96.0"
"aws-cdk.aws-sam" = "^1.96.0"
"aws-cdk.aws-lambda" = "^1.96.0"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Source code for cdk/Makefile:

dev:
    pip install --upgrade pip poetry
    poetry install

Source code for the cdk/cdk.json:

{
  "app": "poetry run python3 app.py",
  "context": {
    "powertools_version": "1.13.0"
  }
}

Source code for the main stack cdk/lambda_stack/lambda_stack.py:

import pathlib

import aws_cdk.aws_lambda as aws_lambda
import aws_cdk.aws_sam as sam
from aws_cdk import core as cdk

class LambdaStack(cdk.Stack):
    def __init__(self, scope: cdk.Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        lambda_src_dir = str(pathlib.Path(__file__).parent.parent.parent) + "/hello-world/"

        # Reference the Layer in a Lambda definition
        aws_lambda.Function(
            self,
            "MyFunction",
            layers=[self.get_powertools_lambda_layer()],
            runtime=aws_lambda.Runtime.PYTHON_3_8,
            handler="app.lambda_handler",
            code=aws_lambda.Code.from_asset(lambda_src_dir),
        )

    def get_powertools_lambda_layer(self):
        powertools_version = self.node.try_get_context("powertools_version")
        powertools_arn = "arn:aws:serverlessrepo:eu-west-1:057560766410:applications/aws-lambda-powertools-python-layer"
        powertools_application = sam.CfnApplication(
            self,
            "AWSLambdaPowertoolsApplication",
            location={
                "applicationId": powertools_arn,
                "semanticVersion": powertools_version,
            },
        )
        powertools_resource = cdk.CfnResource(
            powertools_application,
            "AWSLambdaPowertoolsResource",
            type="AWS::Serverless::Application",
            properties={
                "Location": {
                    "ApplicationId": powertools_arn,
                    "SemanticVersion": powertools_version,
                }
            },
        )
        return aws_lambda.LayerVersion.from_layer_version_arn(
            self,
            "AWSLambdaPowertoolsLayer",
            powertools_resource.get_att("Outputs.LayerVersionArn").to_string(),
        )

[michaelbrewer]

@heitorlessa @am29d ☝🏼 what do you think?

[ran-isenberg]

@michaelbrewer i got the same code in my project but we are using pipenv instead. I'm working on adding a code that will extract the powertools version from the pipenv.lock file and use that for the layer version is the cdk. that way when you upgrade your layer locally, it will also be used automatically for uploaded lambdas layer.

[michaelbrewer]

@michaelbrewer i got the same code in my project but we are using pipenv instead. I'm working on adding a code that will extract the powertools version from the pipenv.lock file and use that for the layer version is the cdk. that way when you upgrade your layer locally, it will also be used automatically for uploaded lambdas layer.

and for Poetry :)

[michaelbrewer]

Coz Python users use: requirements.txt, setup.py, poetry or pipenv for dependencies.

[michaelbrewer]

@risenberg-cyberark i guess self.node.try_get_context("powertools_version") can be abstracted into various choices to get the powertools version

[heitorlessa]

Hey everyone - two quick updates

1. @am29d is on PTO until next week so this will come after 1.14.0 release this week

2. We've decided to publish a public Lambda Layer from our accounts to ease consumption.

We agreed that there are too many customers having challenges with the SAR App approach. Despite providing semantic versioning, these deployment gymnastics to get sem versions for Lambda Layers cause more harm than good whether that is SAM, CDK, Serverless framework etc.

We'll take on these maintenance costs by

1. Publishing a new public Lambda Layer upon new releases
2. Create a dynamic GitHub Badge that shows the latest Lambda Layer available
3. Clean up old Lambda Layers every 6 months (we'll document this)
4. Continue to collect customers feedback to feed in to Lambda Product management team on the need for Lambda Layers Aliases to solve this problem in the long term

We'll start working on that once Alex is back.

Thank you

[ran-isenberg]

@heitorlessa , arent you already publishing a new layer every version?

[heitorlessa]

@heitorlessa , arent you already publishing a new layer every version?

Yes, but it's different. We need additional infrastructure, a cadence for removing old versions without breaking anyone (SAR doesn't as it's local to your account), and a public API to fetch the latest Layer and by version(s).

[ran-isenberg]

@heitorlessa @michaelbrewer @am29d so after i finally got enough dev permissions to use SAR, i came to a realisation that you dont need to deploy both cdk.CfnResource and the sam application in the cdk example. You need only the the sam application. This creates a nested stack with the lambda layerversion resource . Here's the shortened version that creates only 1 nested stack (instead of 2 in the example):

from aws_cdk import core, aws_sam, aws_lambda

class LambdaLayers(core.Construct):

    POWER_TOOLS_VER = '1.13.0'
    POWER_TOOLS_ARN = 'arn:aws:serverlessrepo:eu-west-1:057560766410:applications/aws-lambda-powertools-python-layer-extras'
    POWERTOOLS_BASE_NAME = 'AWSLambdaPowertools'

    def __init__(self, scope: core.Construct, id_: str) -> None:
        super().__init__(scope, id_)
        self.powertools_layer = self._create_powertools_layer()

    def _create_powertools_layer(self: str) -> aws_lambda.LayerVersion:
        powertools_application = aws_sam.CfnApplication(
            self,
            f'{LambdaLayers.POWERTOOLS_BASE_NAME}Application',
            location={
                'applicationId': LambdaLayers.POWER_TOOLS_ARN,
                'semanticVersion': LambdaLayers.POWER_TOOLS_VER
            },
        )

        return aws_lambda.LayerVersion.from_layer_version_arn(
            self,
            f'{LambdaLayers.POWERTOOLS_BASE_NAME}WithExtras',
            powertools_application.get_att("Outputs.LayerVersionArn").to_string(),
        )

[michaelbrewer]

@heitorlessa should we be adding CDK examples along side the AWS Sam instructions?

[heitorlessa]

@michaelbrewer I'd like to wait for more customers to raise before we do. Terraform is more widely used, so I'd like to add all of them as soon as we get our next biggest chunk of work done - Public Lambda Layers ARN by @am29d

[heitorlessa]

Added both CDK and Serverless framework examples - This will be available in the next release. Closing this now and tagging to share when it's released.