Bug: Docker 4.33 + Provided.al2 Image error

[ChronosMasterOfAllTime]

Description:

when trying to run a lambda (either through sam local invoke or sam local start-api

Any invocation of the image is met with the following error

Init failed error=fork/exec /var/task/bootstrap: invalid argument InvokeID=

Steps to reproduce:

Install AWS-SAM-CLI 1.121.0 Install Docker 4.33

Try to execute a lambda using the provided.al2 image

Expectation: Execution succeeds

Actual: Init failed error=fork/exec /var/task/bootstrap: invalid argument InvokeID=

Observed result:

2024-08-07 09:21:09,933 | Using local image: public.ecr.aws/lambda/provided:al2-rapid-x86_64.                                                                              

2024-08-07 09:21:09,935 | Mounting /Users/chronos/REDACTED/bin as /var/task:ro,delegated, inside runtime container               
2024-08-07 09:21:10,297 | Starting a timer for 900 seconds for function 'REDACTED'                                                                                        
2024-08-07 09:21:10,299 | Getting lock for the key localhost-7971                                                                                                          
2024-08-07 09:21:10,300 | Waiting to retrieve the lock (localhost-7971) to start invocation                                                                                
START RequestId: 69028466-59f4-401e-b785-5188b26c4e07 Version: $LATEST
07 Aug 2024 14:21:10,307 [ERROR] (rapid) Init failed error=fork/exec /var/task/bootstrap: invalid argument InvokeID=
2024-08-07 09:21:10,312 | Failed to deserialize response from RIE, returning the raw response as is                                                                        
07 Aug 2024 14:21:10,310 [ERROR] (rapid) Invoke failed error=fork/exec /var/task/bootstrap: invalid argument InvokeID=f21b3f5c-2daf-4002-ba71-30b0c91ea19e
07 Aug 2024 14:21:10,310 [ERROR] (rapid) Invoke DONE failed: Runtime.InvalidEntrypoint

Expected result:

Lambda execution successful, but it isn't.

Execution succeeds with Docker v4.32.0

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

MacOs x86

1. OS: MacOS 13.6.7-x86
2. sam --version: 1.121.0
3. AWS region: us-east-1

{
  "version": "1.121.0",
  "system": {
    "python": "3.12.4",
    "os": "macOS-13.6.7-x86_64-i386-64bit"
  },
  "additional_dependencies": {
    "docker_engine": "27.1.1",
    "aws_cdk": "Not available",
    "terraform": "1.9.3"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}

Add --debug flag to command you are running

[ChronosMasterOfAllTime]

It seems this issue was attempted to be fixed in #7061, however the issue still persists

[ChronosMasterOfAllTime]

Docker 4.33 seems to have caused this issue, downgrading to 4.32 fixed it for now. Can you please look into why?

[mndeveci]

As discussed in the previous issue, this is not originating from SAM CLI, but it is coming from Docker Desktop. What happens if you try to build with SAM_BUILD_MODE=debug sam build, can you do sam local invoke? (Please remember to rebuild without SAM_BUILD_MODE=debug before you actually deploy your function)

[ChronosMasterOfAllTime]

As discussed in the previous issue, this is not originating from SAM CLI, but it is coming from Docker Desktop. What happens if you try to build with SAM_BUILD_MODE=debug sam build, can you do sam local invoke? (Please remember to rebuild without SAM_BUILD_MODE=debug before you actually deploy your function)

I tried that approach and it still didn't work. Cleared my entire docker cache to ensure a "fresh start".

[mndeveci]

Just to confirm you are using a Rust lambda function right?

Docker 4.33 seems to have caused this issue, downgrading to 4.32 fixed it for now.

In this case, I will recommend creating an issue on https://github.com/docker/for-mac/issues for Docker Desktop to track this issue.

[ShepFc3]

Experiencing this with a golang lambda as well on v4.33

[ChronosMasterOfAllTime]

Just to confirm you are using a Rust lambda function right?

Docker 4.33 seems to have caused this issue, downgrading to 4.32 fixed it for now.

In this case, I will recommend creating an issue on https://github.com/docker/for-mac/issues for Docker Desktop to track this issue.

I am running the golang compiled binary on the bootstrap provided.al2023 image as go1.x is deprecated

[ChronosMasterOfAllTime]

Yeah I opened an issue in a couple places. It's either an image issue or docker 4.33 issue

[mndeveci]

Thanks for opening the other issues. Since it works with earlier docker version, I suspect it is related to latest Docker release.

I will keep this open for now for visibility, until we get some updates from the other issues above.

[ChronosMasterOfAllTime]

Fixed in docker 4.34.0

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.