search

aws / aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
https://aws.amazon.com/serverless/sam/
Apache License 2.0
6.51k stars 1.17k forks source link

Can not invoke lambda with custom runtime on local machine (ContentUri -> local file) #1367

Closed fabianfett closed 5 years ago

fabianfett commented 5 years ago

Description

I can not invoke sam local invoke with a layer that points to a local file.

{
  "errorType": "Runtime.ExitError",
  "errorMessage": "RequestId: 52fdfc07-2182-154f-163f-5f0f9a621d72 Error: Couldn't find valid bootstrap(s): [/var/task/bootstrap /opt/bootstrap]"
}

Steps to reproduce

Resources:

  SwiftLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      ContentUri: some-runtime.zip # this results in an error
      ContentUri:                  # this works
        Bucket: some-bucket
        Key: some-runtime.zip

  SquareNumberFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: lambda.zip
      Handler: "SquareNumber.squareNumber"
      Runtime: provided
      Layers:
        - !Ref SwiftLayer
      Events:
        HelloWorld:
          Type: Api
          Properties:
            Path: /hello
            Method: get

Observed result

> echo '{"number": 9 }' | sam local invoke --debug  "SquareNumberFunction"

2019-08-23 17:52:19 Using SAM Template at /Users/fat/Developer/aws-lambda-swift/template.yaml
2019-08-23 17:52:19 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-08-23 17:52:19 Changing event name from before-call.apigateway to before-call.api-gateway
2019-08-23 17:52:19 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-08-23 17:52:19 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-08-23 17:52:19 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-08-23 17:52:19 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-08-23 17:52:19 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-08-23 17:52:19 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-08-23 17:52:19 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-08-23 17:52:19 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-08-23 17:52:19 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-08-23 17:52:19 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-08-23 17:52:19 Changing event name from before-call.apigateway to before-call.api-gateway
2019-08-23 17:52:19 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-08-23 17:52:19 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-08-23 17:52:19 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-08-23 17:52:19 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-08-23 17:52:19 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-08-23 17:52:19 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-08-23 17:52:19 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-08-23 17:52:19 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-08-23 17:52:19 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-08-23 17:52:19 Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2019-08-23 17:52:19 local invoke command is called
2019-08-23 17:52:19 Reading invoke payload from stdin (you can also pass it from file with --event)
2019-08-23 17:52:19 No Parameters detected in the template
2019-08-23 17:52:19 3 resources found in the template
2019-08-23 17:52:19 Found Serverless function with name='SquareNumberFunction' and CodeUri='lambda.zip'
2019-08-23 17:52:19 Trying paths: ['/Users/fat/.docker/config.json', '/Users/fat/.dockercfg']
2019-08-23 17:52:19 Found file at path: /Users/fat/.docker/config.json
2019-08-23 17:52:19 Found 'auths' section
2019-08-23 17:52:19 Auth data for 584525122341.dkr.ecr.eu-central-1.amazonaws.com is absent. Client might be using a credentials store instead.
2019-08-23 17:52:19 Found 'credsStore' section
2019-08-23 17:52:19 http://localhost:None "GET /v1.35/_ping HTTP/1.1" 200 2
2019-08-23 17:52:19 Trying paths: ['/Users/fat/.docker/config.json', '/Users/fat/.dockercfg']
2019-08-23 17:52:19 Found file at path: /Users/fat/.docker/config.json
2019-08-23 17:52:19 Found 'auths' section
2019-08-23 17:52:19 Auth data for 584525122341.dkr.ecr.eu-central-1.amazonaws.com is absent. Client might be using a credentials store instead.
2019-08-23 17:52:19 Found 'credsStore' section
2019-08-23 17:52:19 Found one Lambda function with name 'SquareNumberFunction'
2019-08-23 17:52:19 Invoking SquareNumber.squareNumber (provided)
2019-08-23 17:52:19 No environment variables found for function 'SquareNumberFunction'
2019-08-23 17:52:19 Environment variables overrides data is standard format
2019-08-23 17:52:19 Loading AWS credentials from session with profile 'None'
2019-08-23 17:52:19 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-08-23 17:52:19 Changing event name from before-call.apigateway to before-call.api-gateway
2019-08-23 17:52:19 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-08-23 17:52:19 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-08-23 17:52:19 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-08-23 17:52:19 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-08-23 17:52:19 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-08-23 17:52:19 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-08-23 17:52:19 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-08-23 17:52:19 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-08-23 17:52:19 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-08-23 17:52:19 Looking for credentials via: env
2019-08-23 17:52:19 Looking for credentials via: assume-role
2019-08-23 17:52:19 Looking for credentials via: shared-credentials-file
2019-08-23 17:52:19 Found credentials in shared credentials file: ~/.aws/credentials
2019-08-23 17:52:19 Resolving code path. Cwd=/Users/fat/Developer/aws-lambda-swift, CodeUri=lambda.zip
2019-08-23 17:52:19 Resolved absolute path to code is /Users/fat/Developer/aws-lambda-swift/lambda.zip
2019-08-23 17:52:19 Decompressing /Users/fat/Developer/aws-lambda-swift/lambda.zip
2019-08-23 17:52:19 SwiftLayer is a local Layer in the template
2019-08-23 17:52:19 Resolving code path. Cwd=/Users/fat/Developer/aws-lambda-swift, CodeUri=swift-lambda-runtime.zip
2019-08-23 17:52:19 http://localhost:None "GET /v1.35/images/samcli/lambda:provided-b3ffdccf63b9901f8fdd72d70/json HTTP/1.1" 200 None
2019-08-23 17:52:19 Building image...
2019-08-23 17:52:19 Looking for auth config
2019-08-23 17:52:19 Looking for auth entry for 'https://584525122341.dkr.ecr.eu-central-1.amazonaws.com'
2019-08-23 17:52:19 Sending auth config ('584525122341.dkr.ecr.eu-central-1.amazonaws.com', 'https://584525122341.dkr.ecr.eu-central-1.amazonaws.com')
2019-08-23 17:52:20 http://localhost:None "POST /v1.35/build?t=samcli%2Flambda%3Aprovided-b3ffdccf63b9901f8fdd72d70&q=False&nocache=False&rm=True&forcerm=False&pull=True HTTP/1.1" 200 None
2019-08-23 17:52:22 http://localhost:None "GET /v1.35/images/bb9411a0c2bf/json HTTP/1.1" 200 None
2019-08-23 17:52:22 Trying paths: ['/Users/fat/.docker/config.json', '/Users/fat/.dockercfg']
2019-08-23 17:52:22 Found file at path: /Users/fat/.docker/config.json
2019-08-23 17:52:22 Found 'auths' section
2019-08-23 17:52:22 Auth data for 584525122341.dkr.ecr.eu-central-1.amazonaws.com is absent. Client might be using a credentials store instead.
2019-08-23 17:52:22 Found 'credsStore' section
2019-08-23 17:52:22 http://localhost:None "GET /v1.35/images/samcli/lambda:provided-b3ffdccf63b9901f8fdd72d70/json HTTP/1.1" 200 None
2019-08-23 17:52:22 Requested to skip pulling images ...

2019-08-23 17:52:22 Mounting /private/var/folders/dt/h7ldzyzn209bj6g993wy7p500000gn/T/tmpjz5qi1r4 as /var/task:ro,delegated inside runtime container
2019-08-23 17:52:22 http://localhost:None "POST /v1.35/containers/create HTTP/1.1" 201 88
2019-08-23 17:52:22 http://localhost:None "GET /v1.35/containers/85a86e5fe54d2c73c55a746c43a205df495f1648f59138456a76847bee4c25a3/json HTTP/1.1" 200 None
2019-08-23 17:52:22 http://localhost:None "GET /v1.35/containers/85a86e5fe54d2c73c55a746c43a205df495f1648f59138456a76847bee4c25a3/json HTTP/1.1" 200 None
2019-08-23 17:52:23 http://localhost:None "POST /v1.35/containers/85a86e5fe54d2c73c55a746c43a205df495f1648f59138456a76847bee4c25a3/start HTTP/1.1" 204 0
2019-08-23 17:52:23 Starting a timer for 3 seconds for function 'SquareNumberFunction'
2019-08-23 17:52:23 http://localhost:None "GET /v1.35/containers/85a86e5fe54d2c73c55a746c43a205df495f1648f59138456a76847bee4c25a3/json HTTP/1.1" 200 None
2019-08-23 17:52:23 http://localhost:None "POST /containers/85a86e5fe54d2c73c55a746c43a205df495f1648f59138456a76847bee4c25a3/attach?stdout=1&stderr=1&logs=1&stream=1&stdin=0 HTTP/1.1" 101 0
START RequestId: 52fdfc07-2182-154f-163f-5f0f9a621d72 Version: $LATEST
END RequestId: 52fdfc07-2182-154f-163f-5f0f9a621d72
REPORT RequestId: 52fdfc07-2182-154f-163f-5f0f9a621d72  Duration: 0.02 ms   Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 6 MB   
{
  "errorType": "Runtime.ExitError",
  "errorMessage": "RequestId: 52fdfc07-2182-154f-163f-5f0f9a621d72 Error: Couldn't find valid bootstrap(s): [/var/task/bootstrap /opt/bootstrap]"
}
2019-08-23 17:52:23 http://localhost:None "GET /v1.35/containers/85a86e5fe54d2c73c55a746c43a205df495f1648f59138456a76847bee4c25a3/json HTTP/1.1" 200 None
2019-08-23 17:52:23 http://localhost:None "DELETE /v1.35/containers/85a86e5fe54d2c73c55a746c43a205df495f1648f59138456a76847bee4c25a3?v=False&link=False&force=True HTTP/1.1" 204 0
2019-08-23 17:52:23 Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam local invoke', 'duration': 4760, 'exitReason': 'success', 'exitCode': 0, 'requestId': '4f5293cf-bee0-4834-a977-36fb43bc37d9', 'installationId': '4fa970dc-d89f-4ece-adbb-f117c18e80f7', 'sessionId': '14193f1f-a6e5-4fea-9a0f-04b2d31b1d9a', 'executionEnvironment': 'CLI', 'pyversion': '3.7.4', 'samcliVersion': '0.19.0'}}]}
2019-08-23 17:52:23 Starting new HTTPS connection (1): aws-serverless-tools-telemetry.us-west-2.amazonaws.com:443
2019-08-23 17:52:24 HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)

Expected result

The runtime should be used, as if it was downloaded from S3. Therefore the bootstrap file should be found.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

  1. OS: macOS
  2. sam --version: SAM CLI, version 0.19.0

Maybe this is of interest?

https://www.gitmemory.com/issue/localstack/localstack/1246/487329744

fabianfett commented 5 years ago

This is important to me since I would like to test my runtime in my ci directly after creation. (without upload to S3)

sriram-mv commented 5 years ago

I'm referencing how our integration tests deal with this here: https://github.com/awslabs/aws-sam-cli/blob/develop/tests/integration/testdata/invoke/layers/layer-template.yml#L143

hayd commented 4 years ago

Was this closed because it's fixed? I am still seeing the same issue for provided runtimes.

sebsto commented 4 years ago

@fabianfett if you closed this because you have a solution, can you post the solution here ? Thanks