search

aws-cloudformation / cloudformation-coverage-roadmap

The AWS CloudFormation Public Coverage Roadmap
https://aws.amazon.com/cloudformation/
Creative Commons Attribution Share Alike 4.0 International
1.11k stars 54 forks source link

Global Runtime parameter does not work when using CF Modules #1069

Open wayne-folkes opened 2 years ago

wayne-folkes commented 2 years ago

Name of the resource

Other

Resource Name

AWS::Serverless::Function

Issue Description

When using a CF module in a SAM template the Runtime value specified in the Globals section but not in the AWS::Serverless::Function resource the value appears to be ignored when attempting to deploy and will cause an error to be thrown.

Expected Behavior

The stack should deploy. To get around this issue I have to add the Runtime property to the Function.

If the module is removed from the template the function will deploy as expected when the Runtime value is only specified in the Globals section

Observed Behavior

sam deploy \
                --stack-name wf-module-test-dev \
                --s3-bucket sstk-severless-code-123456789012-us-east-1 \
                --s3-prefix wf-module-test \
                --capabilities CAPABILITY_NAMED_IAM \
                --no-fail-on-empty-changeset \
                --no-progressbar \
                --tags Owner=tech.cloudeng@mycompany.com Team=tech.cloudeng@mycompany.com CostCenter=1525 BusinessUnits=techops Environment=dev Name="Module Test" Application="my-resources" \
                --debug
2022-02-08 16:25:51,366 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2022-02-08 16:25:51,366 | Using config file: samconfig.toml, config environment: default
2022-02-08 16:25:51,366 | Expand command line arguments to:
2022-02-08 16:25:51,366 | --template_file=/Users/wfolkes/dev/alb-lambda/.aws-sam/build/template.yaml --stack_name=wf-module-test-dev --s3_bucket=sstk-severless-code-123456789012-us-east-1 --s3_prefix=wf-module-test --capabilities=('CAPABILITY_NAMED_IAM',) --no_progressbar --tags={'Owner': 'tech.cloudeng@mycompany.com', 'Team': 'tech.cloudeng@mycompany.com', 'CostCenter': '1525', 'BusinessUnits': 'techops', 'Environment': 'dev', 'Name': 'Module Test', 'Application': 'my-resources'} 
2022-02-08 16:25:51,693 | No Parameters detected in the template
2022-02-08 16:25:51,713 | Sam customer defined id is more priority than other IDs. Customer defined id for resource Function is Function
2022-02-08 16:25:51,713 | There is no customer defined id or cdk path defined for resource MyModule, so we will use the resource logical id as the resource id
2022-02-08 16:25:51,714 | Unable to resolve property VpcId: OrderedDict([('Fn::ImportValue', 'shared-vpc-id')]). Leaving as is.
2022-02-08 16:25:51,714 | Unable to resolve property SecurityGroupId: OrderedDict([('Fn::ImportValue', 'shared-vpc-codebuild-default-sg-id')]). Leaving as is.
2022-02-08 16:25:51,714 | Unable to resolve property SubnetIds: OrderedDict([('Fn::ImportValue', 'shared-vpc-private-subnet-ids')]). Leaving as is.
2022-02-08 16:25:51,714 | Unable to resolve property CertificateArn: OrderedDict([('Fn::ImportValue', 'ct-wildcard-cert-arn')]). Leaving as is.
2022-02-08 16:25:51,714 | Unable to resolve property HostedZoneId: OrderedDict([('Fn::ImportValue', 'ct-HostedZoneId')]). Leaving as is.
2022-02-08 16:25:51,714 | Unable to resolve property ZoneId: OrderedDict([('Fn::ImportValue', 'ct-HostedZoneName')]). Leaving as is.
2022-02-08 16:25:51,714 | Unable to resolve property HostName: OrderedDict([('Fn::Sub', ['module-test.${ZoneId}', OrderedDict([('ZoneId', OrderedDict([('Fn::ImportValue', 'ct-HostedZoneName')]))])])]). Leaving as is.
2022-02-08 16:25:51,715 | Unable to resolve property Value: OrderedDict([('Fn::GetAtt', ['MyModuleLoadBalancer', 'DNSName'])]). Leaving as is.
2022-02-08 16:25:51,715 | 2 stacks found in the template
2022-02-08 16:25:51,845 | Sam customer defined id is more priority than other IDs. Customer defined id for resource Function is Function
2022-02-08 16:25:51,845 | There is no customer defined id or cdk path defined for resource MyModule, so we will use the resource logical id as the resource id
2022-02-08 16:25:51,845 | Sam customer defined id is more priority than other IDs. Customer defined id for resource Function is Function
2022-02-08 16:25:52,361 | File with same data already exists at wf-module-test/f28f4bb6f24c12a2984f985b9f45998c, skipping upload
2022-02-08 16:25:52,361 | Sam customer defined id is more priority than other IDs. Customer defined id for resource MyModule is MyModule

        Deploying with following values
        ===============================
        Stack name                   : wf-module-test-dev
        Region                       : us-east-1
        Confirm changeset            : False
        Disable rollback             : False
        Deployment s3 bucket         : sstk-severless-code-123456789012-us-east-1
        Capabilities                 : ["CAPABILITY_NAMED_IAM"]
        Parameter overrides          : {}
        Signing Profiles             : {}

Initiating deployment
=====================
2022-02-08 16:25:52,405 | No Parameters detected in the template
2022-02-08 16:25:52,417 | Sam customer defined id is more priority than other IDs. Customer defined id for resource Function is Function
2022-02-08 16:25:52,417 | Sam customer defined id is more priority than other IDs. Customer defined id for resource MyModule is MyModule
2022-02-08 16:25:52,417 | Unable to resolve property VpcId: OrderedDict([('Fn::ImportValue', 'shared-vpc-id')]). Leaving as is.
2022-02-08 16:25:52,417 | Unable to resolve property SecurityGroupId: OrderedDict([('Fn::ImportValue', 'shared-vpc-codebuild-default-sg-id')]). Leaving as is.
2022-02-08 16:25:52,417 | Unable to resolve property SubnetIds: OrderedDict([('Fn::ImportValue', 'shared-vpc-private-subnet-ids')]). Leaving as is.
2022-02-08 16:25:52,418 | Unable to resolve property CertificateArn: OrderedDict([('Fn::ImportValue', 'ct-wildcard-cert-arn')]). Leaving as is.
2022-02-08 16:25:52,418 | Unable to resolve property HostedZoneId: OrderedDict([('Fn::ImportValue', 'ct-HostedZoneId')]). Leaving as is.
2022-02-08 16:25:52,418 | Unable to resolve property ZoneId: OrderedDict([('Fn::ImportValue', 'ct-HostedZoneName')]). Leaving as is.
2022-02-08 16:25:52,418 | Unable to resolve property HostName: OrderedDict([('Fn::Sub', ['module-test.${ZoneId}', OrderedDict([('ZoneId', OrderedDict([('Fn::ImportValue', 'ct-HostedZoneName')]))])])]). Leaving as is.
2022-02-08 16:25:52,418 | Unable to resolve property Value: OrderedDict([('Fn::GetAtt', ['MyModuleLoadBalancer', 'DNSName'])]). Leaving as is.
2022-02-08 16:25:52,418 | 2 stacks found in the template
2022-02-08 16:25:52,418 | No Parameters detected in the template
2022-02-08 16:25:52,434 | Sam customer defined id is more priority than other IDs. Customer defined id for resource Function is Function
2022-02-08 16:25:52,435 | Sam customer defined id is more priority than other IDs. Customer defined id for resource MyModule is MyModule
2022-02-08 16:25:52,435 | Unable to resolve property VpcId: OrderedDict([('Fn::ImportValue', 'shared-vpc-id')]). Leaving as is.
2022-02-08 16:25:52,435 | Unable to resolve property SecurityGroupId: OrderedDict([('Fn::ImportValue', 'shared-vpc-codebuild-default-sg-id')]). Leaving as is.
2022-02-08 16:25:52,435 | Unable to resolve property SubnetIds: OrderedDict([('Fn::ImportValue', 'shared-vpc-private-subnet-ids')]). Leaving as is.
2022-02-08 16:25:52,435 | Unable to resolve property CertificateArn: OrderedDict([('Fn::ImportValue', 'ct-wildcard-cert-arn')]). Leaving as is.
2022-02-08 16:25:52,435 | Unable to resolve property HostedZoneId: OrderedDict([('Fn::ImportValue', 'ct-HostedZoneId')]). Leaving as is.
2022-02-08 16:25:52,435 | Unable to resolve property ZoneId: OrderedDict([('Fn::ImportValue', 'ct-HostedZoneName')]). Leaving as is.
2022-02-08 16:25:52,435 | Unable to resolve property HostName: OrderedDict([('Fn::Sub', ['module-test.${ZoneId}', OrderedDict([('ZoneId', OrderedDict([('Fn::ImportValue', 'ct-HostedZoneName')]))])])]). Leaving as is.
2022-02-08 16:25:52,436 | Unable to resolve property Value: OrderedDict([('Fn::GetAtt', ['MyModuleLoadBalancer', 'DNSName'])]). Leaving as is.
2022-02-08 16:25:52,436 | 2 resources found in the stack 
2022-02-08 16:25:52,436 | No Parameters detected in the template
2022-02-08 16:25:52,452 | Sam customer defined id is more priority than other IDs. Customer defined id for resource Function is Function
2022-02-08 16:25:52,452 | Sam customer defined id is more priority than other IDs. Customer defined id for resource MyModule is MyModule
2022-02-08 16:25:52,452 | Unable to resolve property VpcId: OrderedDict([('Fn::ImportValue', 'shared-vpc-id')]). Leaving as is.
2022-02-08 16:25:52,452 | Unable to resolve property SecurityGroupId: OrderedDict([('Fn::ImportValue', 'shared-vpc-codebuild-default-sg-id')]). Leaving as is.
2022-02-08 16:25:52,452 | Unable to resolve property SubnetIds: OrderedDict([('Fn::ImportValue', 'shared-vpc-private-subnet-ids')]). Leaving as is.
2022-02-08 16:25:52,452 | Unable to resolve property CertificateArn: OrderedDict([('Fn::ImportValue', 'ct-wildcard-cert-arn')]). Leaving as is.
2022-02-08 16:25:52,453 | Unable to resolve property HostedZoneId: OrderedDict([('Fn::ImportValue', 'ct-HostedZoneId')]). Leaving as is.
2022-02-08 16:25:52,453 | Unable to resolve property ZoneId: OrderedDict([('Fn::ImportValue', 'ct-HostedZoneName')]). Leaving as is.
2022-02-08 16:25:52,453 | Unable to resolve property HostName: OrderedDict([('Fn::Sub', ['module-test.${ZoneId}', OrderedDict([('ZoneId', OrderedDict([('Fn::ImportValue', 'ct-HostedZoneName')]))])])]). Leaving as is.
2022-02-08 16:25:52,453 | Unable to resolve property Value: OrderedDict([('Fn::GetAtt', ['MyModuleLoadBalancer', 'DNSName'])]). Leaving as is.
2022-02-08 16:25:52,976 | File with same data already exists at wf-module-test/f1aea30b5563155b8f0ed3f4fa96bfaf.template, skipping upload

Waiting for changeset to be created..
2022-02-08 16:25:58,645 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': '3a1a43fa-765a-4662-b536-011d2cb9db1d', 'installationId': '351e2a74-4ff1-4af4-919c-cfbe2b0c58bd', 'sessionId': '7c72a6fb-f142-4f8a-8fa2-8a9f4cb82652', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.8.12', 'samcliVersion': '1.37.0', 'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam deploy', 'metricSpecificAttributes': {'projectType': 'CFN'}, 'duration': 7278, 'exitReason': 'ChangeSetError', 'exitCode': 1}}]}
2022-02-08 16:25:58,982 | Telemetry response: 200
Error: Failed to create changeset for the stack: wf-module-test-dev, ex: Waiter ChangeSetCreateComplete failed: Waiter encountered a terminal failure state: For expression "Status" we matched expected path: "FAILED" Status: FAILED. Reason: Transform AWS::Serverless-2016-10-31 failed with: Invalid Serverless Application Specification document. Number of errors found: 1. Resource with id [Function] is invalid. Runtime and Handler needs to be present when PackageType is of type `Zip`
make: *** [deploy] Error 1

Test Cases

A template like the following

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Globals:
  Function:
    Timeout: 3
    Runtime: python3.9

Resources:
  Function:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler

  MyModule:
    Type: 'My::Lambda::ALBFunction::MODULE'
    Properties:
      VpcId: !ImportValue my-vpc-id
      SecurityGroupId: !ImportValue my-vpc-sg-id
      SubnetIds: !ImportValue my-vpc-private-subnet-ids
      CertificateArn: !ImportValue my-wildcard-cert-arn
      FunctionArn: !GetAtt Function.Arn
      HostedZoneId: !ImportValue my-HostedZoneId
      HostName: !Sub
                - module-test.${ZoneId}
                - { ZoneId: !ImportValue my-HostedZoneName }

Other Details

I opened this bug on the SAM project but that was the wrong place for it as this appears to be a Cloudformation issue

https://github.com/aws/serverless-application-model/issues/2318

omnipro-cto commented 2 years ago

Note, a missing Runtime will fail the build but all other function globals are ignored too. Would be great if both issues can be fixed at the same time.

MalikAtalla-AWS commented 1 year ago

CFN Modules currently has the restriction that it can't access any values which are created by a Transform. So, no SAM Global values can be used as input to a Module. We are aware of this issue, but the work to fix it hasn't been scheduled yet. Upvotes to this issue would help my team get this prioritized.

(Note that Modules can be used alongside Transform as long as none of the "Transform values" are fed into Modules)