search

aws / aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
https://aws.amazon.com/serverless/sam/
Apache License 2.0
6.51k stars 1.17k forks source link

!Sub not working for Environment Variable declaration. #1038

Closed ryanpeach closed 5 years ago

ryanpeach commented 5 years ago

Description

When using !Sub, or Fn::Sub, when declaring the environment variables of a lambda function, you get an empty environment variable.

Steps to reproduce

Given a template at /template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description:
Parameters:

  TestParam:
    Type: String
    Default: hello

Resources:

  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello-world/
      Handler: lambda.lambda_handler
      Runtime: python3.6
      Timeout: 3
      Environment:
        Variables:
          DOES_NOT_WORK: !Sub ${TestParam}
          DOES_WORK: !Ref TestParam

And a python function /hello-world/lambda.py

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description:
Parameters:

  TestParam:
    Type: String
    Default: hello

Resources:

  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello-world/
      Handler: lambda.lambda_handler
      Runtime: python3.6
      Timeout: 3
      Environment:
        Variables:
          DOES_NOT_WORK: !Sub ${TestParam}
          DOES_WORK: !Ref TestParam

And an empty requirements.txt file at /hello-world/requirements.txt

And a simple events file at /event.json containing just {}

Run from root:

sam deploy --use-container
sam local invoke HelloWorldFunction --event event.json

Observed result

sam deploy --use-container --debug

2019-03-04 13:03:08 Using SAM Template at /Users/ryapeach/Desktop/sam-app/template.yaml
2019-03-04 13:03:08 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-03-04 13:03:08 Changing event name from before-call.apigateway to before-call.api-gateway
2019-03-04 13:03:08 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-03-04 13:03:08 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-03-04 13:03:08 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-03-04 13:03:08 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-03-04 13:03:08 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-03-04 13:03:08 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-03-04 13:03:08 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-03-04 13:03:08 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-03-04 13:03:08 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-03-04 13:03:08 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-03-04 13:03:08 Changing event name from before-call.apigateway to before-call.api-gateway
2019-03-04 13:03:08 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-03-04 13:03:08 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-03-04 13:03:08 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-03-04 13:03:08 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-03-04 13:03:08 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-03-04 13:03:08 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-03-04 13:03:08 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-03-04 13:03:08 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-03-04 13:03:08 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-03-04 13:03:08 'build' command is called
2019-03-04 13:03:08 Starting Build inside a container
2019-03-04 13:03:08 Collected default values for parameters: {'TestParam': 'hello'}
2019-03-04 13:03:08 1 resources found in the template
2019-03-04 13:03:08 Found Serverless function with name='HelloWorldFunction' and CodeUri='hello-world/'
2019-03-04 13:03:08 Trying paths: ['/Users/ryapeach/.docker/config.json', '/Users/ryapeach/.dockercfg']
2019-03-04 13:03:08 Found file at path: /Users/ryapeach/.docker/config.json
2019-03-04 13:03:08 Found 'auths' section
2019-03-04 13:03:08 Auth data for https://docker-local-scot.artifactory.it.keysight.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for 438307150649.dkr.ecr.us-west-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for https://520713654638.dkr.ecr.us-west-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for 246618743249.dkr.ecr.us-west-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for docker-local-scot.artifactory.it.keysight.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for https://438307150649.dkr.ecr.us-east-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for 438307150649.dkr.ecr.us-east-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for https://438307150649.dkr.ecr.us-west-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for 520713654638.dkr.ecr.us-west-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for https://246618743249.dkr.ecr.us-west-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Found 'credsStore' section
2019-03-04 13:03:08 Building resource 'HelloWorldFunction'
2019-03-04 13:03:08 Trying paths: ['/Users/ryapeach/.docker/config.json', '/Users/ryapeach/.dockercfg']
2019-03-04 13:03:08 Found file at path: /Users/ryapeach/.docker/config.json
2019-03-04 13:03:08 Found 'auths' section
2019-03-04 13:03:08 Auth data for https://docker-local-scot.artifactory.it.keysight.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for 438307150649.dkr.ecr.us-west-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for https://520713654638.dkr.ecr.us-west-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for 246618743249.dkr.ecr.us-west-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for docker-local-scot.artifactory.it.keysight.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for https://438307150649.dkr.ecr.us-east-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for 438307150649.dkr.ecr.us-east-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for https://438307150649.dkr.ecr.us-west-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for 520713654638.dkr.ecr.us-west-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Auth data for https://246618743249.dkr.ecr.us-west-2.amazonaws.com is absent. Client might be using a credentials store instead.
2019-03-04 13:03:08 Found 'credsStore' section
2019-03-04 13:03:08 http://localhost:None "GET /v1.35/images/lambci/lambda:build-python3.6/json HTTP/1.1" 200 None
2019-03-04 13:03:08 Looking for auth config
2019-03-04 13:03:08 Using credentials store "osxkeychain"
2019-03-04 13:03:08 Looking for auth entry for 'https://index.docker.io/v1/'
2019-03-04 13:03:08 No entry found
2019-03-04 13:03:08 No entry in credstore - fetching from auth dict
2019-03-04 13:03:08 Looking for auth entry for 'docker.io'
2019-03-04 13:03:08 No entry found
2019-03-04 13:03:08 No auth config found
2019-03-04 13:03:08 http://localhost:None "POST /v1.35/images/create?tag=build-python3.6&fromImage=lambci%2Flambda HTTP/1.1" 200 None

Fetching lambci/lambda:build-python3.6 Docker container image......
2019-03-04 13:03:08 Mounting /Users/ryapeach/Desktop/sam-app/hello-world as /tmp/samcli/source:ro inside runtime container
2019-03-04 13:03:08 http://localhost:None "POST /v1.35/containers/create HTTP/1.1" 201 90
2019-03-04 13:03:08 http://localhost:None "GET /v1.35/containers/eebef2d69a83f927e7f7ca8d6bac2f27193c7160dba5ebf4b1930e4222a7226e/json HTTP/1.1" 200 None
2019-03-04 13:03:08 http://localhost:None "GET /v1.35/containers/eebef2d69a83f927e7f7ca8d6bac2f27193c7160dba5ebf4b1930e4222a7226e/json HTTP/1.1" 200 None
2019-03-04 13:03:09 http://localhost:None "POST /v1.35/containers/eebef2d69a83f927e7f7ca8d6bac2f27193c7160dba5ebf4b1930e4222a7226e/start HTTP/1.1" 204 0
2019-03-04 13:03:09 http://localhost:None "GET /v1.35/containers/eebef2d69a83f927e7f7ca8d6bac2f27193c7160dba5ebf4b1930e4222a7226e/json HTTP/1.1" 200 None
2019-03-04 13:03:09 http://localhost:None "POST /containers/eebef2d69a83f927e7f7ca8d6bac2f27193c7160dba5ebf4b1930e4222a7226e/attach?stdout=1&stderr=1&logs=1&stream=1&stdin=0 HTTP/1.1" 101 0
Using the request object from command line argument
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Running workflow 'PythonPipBuilder'
Running PythonPipBuilder:ResolveDependencies
PythonPipBuilder:ResolveDependencies succeeded
Running PythonPipBuilder:CopySource
PythonPipBuilder:CopySource succeeded
2019-03-04 13:03:09 Build inside container returned response {"jsonrpc": "2.0", "id": 1, "result": {"artifacts_dir": "/tmp/samcli/artifacts"}}
2019-03-04 13:03:09 Build inside container was successful. Copying artifacts from container to host
2019-03-04 13:03:09 http://localhost:None "GET /v1.35/containers/eebef2d69a83f927e7f7ca8d6bac2f27193c7160dba5ebf4b1930e4222a7226e/json HTTP/1.1" 200 None
2019-03-04 13:03:09 Copying from container: /tmp/samcli/artifacts/. -> /Users/ryapeach/Desktop/sam-app/.aws-sam/build/HelloWorldFunction
2019-03-04 13:03:09 http://localhost:None "GET /v1.35/containers/eebef2d69a83f927e7f7ca8d6bac2f27193c7160dba5ebf4b1930e4222a7226e/archive?path=%2Ftmp%2Fsamcli%2Fartifacts%2F. HTTP/1.1" 200 480
2019-03-04 13:03:09 http://localhost:None "GET /v1.35/containers/eebef2d69a83f927e7f7ca8d6bac2f27193c7160dba5ebf4b1930e4222a7226e/json HTTP/1.1" 200 None
2019-03-04 13:03:09 http://localhost:None "DELETE /v1.35/containers/eebef2d69a83f927e7f7ca8d6bac2f27193c7160dba5ebf4b1930e4222a7226e?v=False&link=False&force=True HTTP/1.1" 204 0
2019-03-04 13:03:09 Build inside container succeeded

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Package: sam package --s3-bucket <yourbucket>

sam local invoke HelloWorldFunction --event event.json --debug

2019-03-04 13:02:37 Found credentials in shared credentials file: ~/.aws/credentials
2019-03-04 13:02:37 Invoking lambda.lambda_handler (python3.6)

Fetching lambci/lambda:python3.6 Docker container image......
2019-03-04 13:02:38 Mounting /Users/ryapeach/Desktop/sam-app/.aws-sam/build/HelloWorldFunction as /var/task:ro inside runtime container
START RequestId: 3f5f0436-8750-4a13-a558-6bd585200200 Version: $LATEST
'DOES_NOT_WORK' value is: 
'DOES_WORK' value is: hello
END RequestId: 3f5f0436-8750-4a13-a558-6bd585200200
REPORT RequestId: 3f5f0436-8750-4a13-a558-6bd585200200 Duration: 14 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 19 MB

The important part:

'DOES_NOT_WORK' value is: 
'DOES_WORK' value is: hello

Expected result

'DOES_NOT_WORK' value is: hello
'DOES_WORK' value is: hello

DOES_NOT_WORK did not have a value, but it did exist in the environment variables.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

  1. OS:

macOS High Sierra Version 10.13.6

  1. sam --version:

SAM CLI, version 0.12.0

ryanpeach commented 5 years ago

Same for some other functions like Fn::Join

jfuss commented 5 years ago

Many of the intrinsics are not currently supported and this is a duplicate of #528

Closing

joshpeng commented 5 years ago

@ryanpeach How did you even get !Ref to work? I am receiving this error:

Invalid value in Template for key: MY_ENV_VAR: expected string

When I try this in my template.yml:

Parameters:
  SomeTestParam:
    Type: String
    Default: foobar

...

Environment:
    Variables:
        MY_ENV_VAR: !Ref SomeTestParam

I am on SAM CLI v0.19.0.

shailesh-kanzariya commented 4 years ago

Use !Sub when you want to combine/concatenate 'parameter' with some other string e.g. !Sub "this-is-${parameter}" When you want to use the parameter only, use it as !Ref e.g. !Ref environment In above, 'environment' is declared as 'parameter' and passed with '--parameter-overrides'