(cli): Hotswapping support for Lambda function's description and or environment variable

huyphan commented 2 years ago

Describe the feature

Lambda users are hitting a situation with hotswap deployment where either (1) hotswap deployment is always skipped or (2) it is not compatible with subsequent full deployments.

Use Case

The issue happens with a setup where:

A Lambda function is using S3 code provider (such as lambda.Code.fromBucket)
The Lambda function deployment is managed with Versions and Aliases.
Users need to be able to deploy their stack properly, either with or without --hotswap directive.

For that setup, we suggest in our doc that users should define a lambda.Version directly:

NOTE: The currentVersion property is only supported when your AWS Lambda function uses either lambda.Code.fromAsset or lambda.Code.fromInline. Other types of code providers (such as lambda.Code.fromBucket) require that you define a lambda.Version resource directly since the CDK is unable to determine if their contents had changed.

We also say that we should apply some trick to the Function resource's properties so the Version can be updated properly

One way to ensure that the lambda.Version always points to the latest version of your lambda.Function is to set an environment variable which changes at least as often as your code does. This makes sure the function always has the latest code.

There are at least three ways of doing it with CDK, but every of them gives users error at some point in their development cycle.

Option 1: Adding a dynamic environment variable

This option is suggested in the official doc. An example code looks like this:

    const lambdaFunction = new Function(this, "TestFunction", {
      code: Code.fromBucket(
        Bucket.fromBucketName(
          this,
          "LambdaCodeBucket",
          "teststack-testbucket560b80bc-1vg8000mxtcs0"
        ),
        "lambda.zip"
      ),
      runtime: Runtime.PYTHON_3_9,
      handler: "index.handler",
      environment: {
        CodeVersionString: new Date().toISOString(),
      },
    });

Since Function's environment update is not considered hotswappable, user can never perform a hotswap deployment with this code. In my local setup where the hotswap deployment was skipped, this was the diff for this case:

Stack LambdaStack
Resources
[~] AWS::Lambda::Function TestFunction TestFunction22AD90FC
 └─ [~] Environment
     └─ [~] .Variables:
         └─ [~] .CodeVersionString:
             ├─ [-] 2022-06-18T11:14:03.458Z
             └─ [+] 2022-06-18T11:46:41.225Z

Option 2: Adding a dynamic value to the function description

If I recall correctly, one example in AWS CDK docs was using this approach, but that snippet is now deleted. But this is a commonly known trick to mutate the Lambda function's properties:

    const lambdaFunction = new Function(this, "TestFunction", {
      code: Code.fromBucket(
        Bucket.fromBucketName(
          this,
          "LambdaCodeBucket",
          "teststack-testbucket560b80bc-1vg8000mxtcs0"
        ),
        "lambda.zip"
      ),
      runtime: Runtime.PYTHON_3_9,
      handler: "index.handler",
      description: "generated at " + new Date().toISOString(),  
    });

This option has the same problem as option #1 -- hotswap deployment is always skipped because an update of a Lambda function's description is not a hotswappable change.

Option 3: Making the Version resource dynamic instead

This option was used as an example by the CDK folks in a comment on a similar topic in the past:

    const lambdaFunction = new Function(this, "TestFunction", {
      code: Code.fromBucket(
        Bucket.fromBucketName(this, "LambdaCodeBucket", "teststack-testbucket560b80bc-1vg8000mxtcs0"),
        "lambda.zip"
      ),
      runtime: Runtime.PYTHON_3_9,
      handler: "index.handler",
    });

    const version = new Version(this, `TestFunctionVersion-${new Date().toISOString()}`, {
      lambda: lambdaFunction
    });

This trick does allow hotswap deployment to work, but will cause every subsequent full deployment to fail because of conflicting function version. Example error message:

7:46:27 PM | CREATE_FAILED        | AWS::Lambda::Version  | TestFunctionVersio...094605709ZC17DD4A4
A version for this Lambda function exists ( 1 ). Modify the function to create a new version.

Proposed Solution

If we could make option 3 works, that would be ideal. However, it require users to have access to a value that changes consistently with the Lambda code (such as the hash of the Lambda code or the local build version). And that value has to be available at synthesis time. This is not feasible for many users.

So we are left with the options of making Lambda function's descriptions and/or environment variables hotswappable. That said, there may be better options that I haven't thought of yet.

Other Information

No response

Acknowledgements

[X] I may be able to implement this feature request
[ ] This feature might incur a breaking change

CDK version used

2.28.1 (build d035432)

Environment details (OS name and version, etc.)

Darwin Kernel Version 21.5.0 . root:xnu-8020.121.3~4/RELEASE_X86_64 x86_64

github-actions[bot] commented 2 years ago

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

mrgrain commented 2 years ago

PR needed to be reverted due to a failure with integration tests https://github.com/aws/aws-cdk/pull/21509

github-actions[bot] commented 2 years ago

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

aws / aws-cdk