DeploymentAwaiter fails to update in a project with serverless containers enabled

ashphy commented 1 month ago

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

v20.13.1

Amplify CLI Version

12.12.4

What operating system are you using?

macOS Sonoma 14.5

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No manual changes made

Describe the bug

In a project with Serverless containers enabled, updating a backend deployed with Amplify CLI v12.12.3 or earlier to Amplify CLI v12.12.4 and then running amplify push results in a failure to update DeploymentAwaiter.

Context:

DeploymentAwaiter is deployed using a CloudFormation Custom Resource.
Previously ran on Node 16, changed to Node 18 in #2529.
Node 18 runtime image does not include aws-sdk v2, requiring updated source code aws-amplify/amplify-cli#13757.

Issue:

The runtime is updated to Node 18, but the Lambda source code is not updated during amplify push, causing the following error:

INIT_START Runtime Version: nodejs:18.v31   Runtime Version ARN: arn:aws:lambda:us-west-2::runtime:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
2024-07-30T12:16:16.780Z    undefined   ERROR   Uncaught Exception  {
    "errorType": "Runtime.ImportModuleError",
    "errorMessage": "Error: Cannot find module 'aws-sdk'\nRequire stack:\n- /var/task/outbound.js\n- /var/task/cfn-response.js\n- /var/task/framework.js\n- /var/runtime/index.mjs",
    "stack": [
        "Runtime.ImportModuleError: Error: Cannot find module 'aws-sdk'",
        "Require stack:",
        "- /var/task/outbound.js",
        "- /var/task/cfn-response.js",
        "- /var/task/framework.js",
        "- /var/runtime/index.mjs",
        "    at _loadUserApp (file:///var/runtime/index.mjs:1087:17)",
        "    at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1119:21)",
        "    at async start (file:///var/runtime/index.mjs:1282:23)",
        "    at async file:///var/runtime/index.mjs:1288:1"
    ]
}

Reviewing the CloudFormation Template reveals the following entry:

File: amplify/backend/api/containera76cba34/containera76cba34-cloudformation-template.json

"AwaiterMyProviderframeworkonEventB703D418": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
  "S3Bucket": {
    "Ref": "deploymentBucketName"
  },
  "S3Key": {
    "Ref": "awaiterS3Key"
  }
},
"Description": "AWS CDK resource provider framework - onEvent (ContainersStack/AwaiterMyProvider)",

Code: https://github.com/aws-amplify/amplify-category-api/blob/2e75a79816da2f5c928bdaa900f9385b96fd5581/packages/amplify-category-api/src/provider-utils/awscloudformation/base-api-stack.ts#L542-L545

The S3Key is fixed, which likely causes the source code not to be updated.

Changes to a deployment package in Amazon S3 or a container image in ECR are not detected automatically during stack updates. To update the function code, change the object key or version in the template.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-lambda-function-code.html

The source code custom-resource-pipeline-awaiter.zip uploaded to the deployment bucket has been confirmed to be updated. However, the code on the lambda was confirmed to be NOT updated.

Expected behavior

The DeploymentAwaiter source code should be updated to the latest version, and CloudFormation should complete successfully.

Reproduction steps

Create an Amplify app with Amplify CLI v12.12.3 and enable Serverless containers.
Run amplify push to update the backend.
Update Amplify CLI to v12.12.4.
Run amplify push to update the backend.
An error occurs.

Project Identifier

Project Identifier: 52a166481f025be404045fcd549de961

Log output

No response

Additional information

No response

Before submitting, please confirm:

[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

AnilMaktala commented 1 month ago

Hey @ashphy, Thank you for raising this issue. I am working on reproducing it and will keep you updated on the outcome.

AnilMaktala commented 1 month ago

Hey @ashphy, We can reproduce the issue by following the steps mentioned in the description. Hence, marking this as bug for the team to evaluate further. To unblock, could you please downgrade to the previous CLI version until we resolve this issue?

ashphy commented 1 month ago

Thank you for your prompt response. As suggested, I will downgrade to the previous version. Please feel free to close the issue once the fix is merged.

Jupdi commented 1 month ago

We are also affected by the error and would appreciate a fast fix.

dpilch commented 1 month ago

CLI v12.12.6 has been released with the fix.

github-actions[bot] commented 1 month ago

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.

aws-amplify / amplify-category-api