aws / aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
https://aws.amazon.com/serverless/sam/
Apache License 2.0
6.5k stars 1.17k forks source link

Number not being honored when passed as parameter to sam local start-lambda #925

Closed binarymist closed 4 years ago

binarymist commented 5 years ago

Description

--parameter-overrides number not honored

Steps to reproduce

Run: sam local start-lambda --parameter-overrides ParameterKey=TimeoutParameter,ParameterValue=3 --host 172.25.0.1 --docker-network compose_pt-net

Observed result

2019-01-11 00:28:51 local start_lambda command is called
2019-01-11 00:28:51 Collected default values for parameters: {'TimeoutParameter': 10}
2019-01-11 00:28:51 2 resources found in the template
2019-01-11 00:28:51 Found Serverless function with name='provisionAppSlaves' and CodeUri='./app-slave-provisioner'
2019-01-11 00:28:51 Found Serverless function with name='provisionSeleniumStandalones' and CodeUri='./selenium-standalone-provisioner'
2019-01-11 00:28:51 Trying paths: ['.docker/config.json', '.dockercfg']
2019-01-11 00:28:51 No config file found
2019-01-11 00:28:51 Trying paths: ['.docker/config.json', '.dockercfg']
2019-01-11 00:28:51 No config file found
2019-01-11 00:28:51 http://localhost:None "GET /v1.35/_ping HTTP/1.1" 200 2
2019-01-11 00:28:51 Trying paths: ['.docker/config.json', '.dockercfg']
2019-01-11 00:28:51 No config file found
2019-01-11 00:28:51 Trying paths: ['.docker/config.json', '.dockercfg']
2019-01-11 00:28:51 No config file found
2019-01-11 00:28:51 Starting the Local Lambda Service. You can now invoke your Lambda Functions defined in your template through the endpoint.
2019-01-11 00:28:51 Localhost server is starting up. Multi-threading = True
2019-01-11 00:28:51  * Running on http://172.25.0.1:3001/ (Press CTRL+C to quit)

When I invoke the lambda via the aws-sdk in code, sam local says:

2019-01-11 00:29:55 Found one Lambda function with name 'provisionAppSlaves'
2019-01-11 00:29:55 Found one Lambda function with name 'provisionSeleniumStandalones'
2019-01-11 00:29:55 Invoking index.provisionAppSlaves (nodejs8.10)
2019-01-11 00:29:55 Invoking index.provisionSeleniumStandalones (nodejs8.10)
2019-01-11 00:29:55 Environment variables overrides data is standard format
2019-01-11 00:29:55 Environment variables overrides data is standard format
2019-01-11 00:29:55 Loading AWS credentials from session with profile 'None'
2019-01-11 00:29:55 Loading AWS credentials from session with profile 'None'
2019-01-11 00:29:55 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-01-11 00:29:55 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-01-11 00:29:55 Changing event name from before-call.apigateway to before-call.api-gateway
2019-01-11 00:29:55 Changing event name from before-call.apigateway to before-call.api-gateway
2019-01-11 00:29:55 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-01-11 00:29:55 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-01-11 00:29:55 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-01-11 00:29:55 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-01-11 00:29:55 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-01-11 00:29:55 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-01-11 00:29:55 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-01-11 00:29:55 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-01-11 00:29:55 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-01-11 00:29:55 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-01-11 00:29:55 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-01-11 00:29:55 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-01-11 00:29:55 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-01-11 00:29:55 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-01-11 00:29:55 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-01-11 00:29:55 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-01-11 00:29:55 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-01-11 00:29:55 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-01-11 00:29:55 Looking for credentials via: env
2019-01-11 00:29:55 Looking for credentials via: assume-role
2019-01-11 00:29:55 Looking for credentials via: shared-credentials-file
2019-01-11 00:29:55 Found credentials in shared credentials file: ~/.aws/credentials
2019-01-11 00:29:55 Resolving code path. Cwd=/Source/purpleteam-lambda, CodeUri=./selenium-standalone-provisioner
2019-01-11 00:29:55 Resolved absolute path to code is /Source/purpleteam-lambda/selenium-standalone-provisioner
2019-01-11 00:29:55 Code /Source/purpleteam-lambda/selenium-standalone-provisioner is not a zip/jar file
2019-01-11 00:29:55 Trying paths: ['.docker/config.json', '.dockercfg']
2019-01-11 00:29:55 No config file found
2019-01-11 00:29:55 Trying paths: ['.docker/config.json', '.dockercfg']
2019-01-11 00:29:55 No config file found
2019-01-11 00:29:55 http://localhost:None "GET /v1.35/images/lambci/lambda:nodejs8.10/json HTTP/1.1" 200 None
2019-01-11 00:29:55 Looking for auth config
2019-01-11 00:29:55 No auth config in memory - loading from filesystem
2019-01-11 00:29:55 Trying paths: ['.docker/config.json', '.dockercfg']
2019-01-11 00:29:55 No config file found
2019-01-11 00:29:55 Looking for auth entry for 'docker.io'
2019-01-11 00:29:55 No entry found
2019-01-11 00:29:55 No auth config found
2019-01-11 00:29:55 Looking for credentials via: env
2019-01-11 00:29:55 Looking for credentials via: assume-role
2019-01-11 00:29:55 Looking for credentials via: shared-credentials-file
2019-01-11 00:29:55 Found credentials in shared credentials file: ~/.aws/credentials
2019-01-11 00:29:55 Resolving code path. Cwd=/Source/purpleteam-lambda, CodeUri=./app-slave-provisioner
2019-01-11 00:29:55 Resolved absolute path to code is /Source/purpleteam-lambda/app-slave-provisioner
2019-01-11 00:29:55 Code /Source/purpleteam-lambda/app-slave-provisioner is not a zip/jar file
2019-01-11 00:29:55 Trying paths: ['.docker/config.json', '.dockercfg']
2019-01-11 00:29:55 No config file found
2019-01-11 00:29:55 Trying paths: ['.docker/config.json', '.dockercfg']
2019-01-11 00:29:55 No config file found
2019-01-11 00:29:55 http://localhost:None "GET /v1.35/images/lambci/lambda:nodejs8.10/json HTTP/1.1" 200 None
2019-01-11 00:29:55 Looking for auth config
2019-01-11 00:29:55 No auth config in memory - loading from filesystem
2019-01-11 00:29:55 Trying paths: ['.docker/config.json', '.dockercfg']
2019-01-11 00:29:55 No config file found
2019-01-11 00:29:55 Looking for auth entry for 'docker.io'
2019-01-11 00:29:55 No entry found
2019-01-11 00:29:55 No auth config found
2019-01-11 00:29:58 http://localhost:None "POST /v1.35/images/create?tag=nodejs8.10&fromImage=lambci%2Flambda HTTP/1.1" 200 None

Fetching lambci/lambda:nodejs8.10 Docker container image......
2019-01-11 00:29:58 Mounting /Source/purpleteam-lambda/selenium-standalone-provisioner as /var/task:ro inside runtime container
2019-01-11 00:29:58 http://localhost:None "POST /v1.35/images/create?tag=nodejs8.10&fromImage=lambci%2Flambda HTTP/1.1" 200 None

Fetching lambci/lambda:nodejs8.10 Docker container image......
2019-01-11 00:29:58 Mounting /Source/purpleteam-lambda/app-slave-provisioner as /var/task:ro inside runtime container
2019-01-11 00:29:58 http://localhost:None "POST /v1.35/containers/create HTTP/1.1" 201 201
2019-01-11 00:29:58 http://localhost:None "GET /v1.35/containers/156c0040698b3e7be8d314baf595b8f3882711184e9dfb900a05e72662c5990d/json HTTP/1.1" 200 None
2019-01-11 00:29:58 http://localhost:None "GET /v1.35/networks/compose_pt-net HTTP/1.1" 200 None
2019-01-11 00:29:58 http://localhost:None "POST /v1.35/containers/create HTTP/1.1" 201 201
2019-01-11 00:29:58 http://localhost:None "GET /v1.35/containers/7aa23728a90bcd90a5155579521635a953205a1345f57a00bab825c6fd1ff0a9/json HTTP/1.1" 200 None
2019-01-11 00:29:58 http://localhost:None "GET /v1.35/networks/compose_pt-net HTTP/1.1" 200 None
2019-01-11 00:29:58 http://localhost:None "POST /v1.35/networks/a4f1162d57b3f5aff37729656ef2a506c27a264cd7665ec0703a4ea9334f988c/connect HTTP/1.1" 200 0
2019-01-11 00:29:58 http://localhost:None "GET /v1.35/containers/156c0040698b3e7be8d314baf595b8f3882711184e9dfb900a05e72662c5990d/json HTTP/1.1" 200 None
2019-01-11 00:29:58 http://localhost:None "POST /v1.35/networks/a4f1162d57b3f5aff37729656ef2a506c27a264cd7665ec0703a4ea9334f988c/connect HTTP/1.1" 200 0
2019-01-11 00:29:58 http://localhost:None "GET /v1.35/containers/7aa23728a90bcd90a5155579521635a953205a1345f57a00bab825c6fd1ff0a9/json HTTP/1.1" 200 None
2019-01-11 00:29:59 http://localhost:None "POST /v1.35/containers/156c0040698b3e7be8d314baf595b8f3882711184e9dfb900a05e72662c5990d/start HTTP/1.1" 204 0
2019-01-11 00:29:59 Starting a timer for 3 seconds for function 'provisionSeleniumStandalones'
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 1071, in run
    self.finished.wait(self.interval)
  File "/usr/lib/python2.7/threading.py", line 614, in wait
    self.__cond.wait(timeout)
  File "/usr/lib/python2.7/threading.py", line 349, in wait
    endtime = _time() + timeout
TypeError: unsupported operand type(s) for +: 'float' and 'str'

2019-01-11 00:29:59 http://localhost:None "GET /v1.35/containers/156c0040698b3e7be8d314baf595b8f3882711184e9dfb900a05e72662c5990d/json HTTP/1.1" 200 None
2019-01-11 00:29:59 http://localhost:None "POST /containers/156c0040698b3e7be8d314baf595b8f3882711184e9dfb900a05e72662c5990d/attach?stream=1&stdin=0&logs=1&stderr=1&stdout=1 HTTP/1.1" 101 0
START RequestId: 4de742e6-2284-13c9-520f-fb2996acc87e Version: $LATEST
2019-01-11 00:29:59 http://localhost:None "POST /v1.35/containers/7aa23728a90bcd90a5155579521635a953205a1345f57a00bab825c6fd1ff0a9/start HTTP/1.1" 204 0
2019-01-11 00:29:59 Starting a timer for 3 seconds for function 'provisionAppSlaves'
Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 1071, in run
    self.finished.wait(self.interval)
  File "/usr/lib/python2.7/threading.py", line 614, in wait
    self.__cond.wait(timeout)
  File "/usr/lib/python2.7/threading.py", line 349, in wait
    endtime = _time() + timeout
TypeError: unsupported operand type(s) for +: 'float' and 'str'

2019-01-11 00:29:59 http://localhost:None "GET /v1.35/containers/7aa23728a90bcd90a5155579521635a953205a1345f57a00bab825c6fd1ff0a9/json HTTP/1.1" 200 None
2019-01-11 00:29:59 http://localhost:None "POST /containers/7aa23728a90bcd90a5155579521635a953205a1345f57a00bab825c6fd1ff0a9/attach?stream=1&stdin=0&logs=1&stderr=1&stdout=1 HTTP/1.1" 101 0
START RequestId: c591f426-93b2-1683-1de2-db6671d97edc Version: $LATEST
END RequestId: c591f426-93b2-1683-1de2-db6671d97edc
REPORT RequestId: c591f426-93b2-1683-1de2-db6671d97edc  Duration: 85.55 ms  Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 39 MB  
2019-01-11 00:30:00 http://localhost:None "GET /v1.35/containers/7aa23728a90bcd90a5155579521635a953205a1345f57a00bab825c6fd1ff0a9/json HTTP/1.1" 200 None
2019-01-11 00:30:00 http://localhost:None "DELETE /v1.35/containers/7aa23728a90bcd90a5155579521635a953205a1345f57a00bab825c6fd1ff0a9?force=True&link=False&v=False HTTP/1.1" 204 0

Expected result

Expect the ParameterValue=3 to be honored (timeout the lambda function when invoked within 3 seonds) within the template.yaml when the lambda function is finally invoked.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

  1. OS: GNU/Linux
  2. sam --version: 0.6.2

Relevant part of template.yaml:

Parameters:
  TimeoutParameter:
    Type: Number
    Default: 10

Globals:

  Function:
    Timeout: !Ref TimeoutParameter
    Runtime: nodejs8.10
    Environment:
      Variables:
          NODE_ENV: development

Resources:

  provisionAppSlaves:
      Type: AWS::Serverless::Function
      Properties:
          CodeUri: ./app-slave-provisioner
          Handler: index.provisionAppSlaves

  provisionSeleniumStandalones:
      Type: AWS::Serverless::Function
      Properties:
          CodeUri: ./selenium-standalone-provisioner
          Handler: index.provisionSeleniumStandalones

The Default: 10 Timeout is only honored If the sam local command leaves out the --parameter-overrides ParameterKey=TimeoutParameter,ParameterValue=3

binarymist commented 5 years ago

If I try:

Globals:
  Function:
    Timeout: !Ref LAMBDA_TIMEOUT # Default is 3 seconds
    Runtime: nodejs8.10
    Environment:
      Variables: # More info about Env Vars: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#environment-object
        NODE_ENV: development
        LAMBDA_TIMEOUT: 5 # Set the env var here, then ref it in code and subtract a little for internal timeout. Could also reference the env var from convict file.

Resources:

  provisionAppSlaves:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: ./app-slave-provisioner
      Handler: index.provisionAppSlaves

  provisionSeleniumStandalones:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: ./selenium-standalone-provisioner
      Handler: index.provisionSeleniumStandalones

Outputs:

  provisionAppSlaves:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt provisionAppSlaves.Arn

  provisionAppSlavesIamRole:
    Description: "Implicit IAM Role created for provisionAppSlaves function"
    Value: !GetAtt provisionAppSlavesRole.Arn

The !Ref LAMBDA_TIMEOUT doesn't work here. sam local start-lambda ... says:

2019-01-13 18:43:39 Starting a timer for {u'Ref': u'LAMBDA_TIMEOUT'} seconds for function 'provisionAppSlaves'
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 1071, in run
    self.finished.wait(self.interval)
  File "/usr/lib/python2.7/threading.py", line 614, in wait
    self.__cond.wait(timeout)
  File "/usr/lib/python2.7/threading.py", line 349, in wait
    endtime = _time() + timeout
TypeError: unsupported operand type(s) for +: 'float' and 'dict'

So we have to duplicate the value like so:

Globals:
  Function:
    Timeout: 5 # !Ref LAMBDA_TIMEOUT # Default is 3 seconds # !Ref doesn't work here, so we have to duplicate the value.
    Runtime: nodejs8.10
    Environment:
      Variables: # More info about Env Vars: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#environment-object
        NODE_ENV: development
        LAMBDA_TIMEOUT: 5 # Set the env var here, then ref it in code and subtract a little for internal timeout. Could also reference the env var from convict file.
isaiahgrant commented 5 years ago

This issue does not appear to be limited to the number type.

I have a template that utilizes intrinsic functions and Conditionals to define behavior for different environments. When deployed to AWS the functionality works as intended however with SAM Local the defaults are used and any parameter-override values appear to be ignored.

For anyone else experiencing this issue you can easily override the default values for testing. Of course in a local environment it defeats the purpose of the parameters :disappointed:, but it will keep you going while this hangs around (in the Backlog...).

Also if you are using Lambdas you have the alternative option of overriding environment variables directly (https://github.com/awslabs/aws-sam-cli/blob/develop/docs/advanced_usage.md#environment-variable-file). This doesn't accomplish all the utility of the template parameters but it may help!

As an aside SAM Local appears to ignore the conditionals entirely so careful when testing locally (https://github.com/awslabs/aws-sam-cli/issues/194).

jfuss commented 4 years ago

This was released a long time ago in v0.30.0.

Closing