aws / serverless-application-model

The AWS Serverless Application Model (AWS SAM) transform is a AWS CloudFormation macro that transforms SAM templates into CloudFormation templates.
https://aws.amazon.com/serverless/sam
Apache License 2.0
9.33k stars 2.38k forks source link

CodeUri overriding InlineCode #1820

Closed dehli closed 3 years ago

dehli commented 3 years ago

Description:

Steps to reproduce the issue:

  1. Create an AWS::Serverless::Function with both InlineCode and CodeUri properties set.

Observed result: CodeUri is uploaded to the lambda function

Function:
  Type: AWS::Serverless::Function
  Properties:
    AutoPublishAlias: live
    Handler: index.handler
    Role:
      Fn::GetAtt:
      - Role
      - Arn
    Runtime: nodejs10.x
    InlineCode: "code"
    CodeUri: s3://...

is processed into:

"Function": {
  "Type": "AWS::Lambda::Function",
  "Properties": {
    "Handler": "index.handler",
    "Code": {
      "S3Bucket": "bucket",
      "S3Key": "key"
    },
    "Role": {
      "Fn::GetAtt": [
        "Role",
        "Arn"
      ]
    },
    "Runtime": "nodejs10.x",
    "Tags": [
      {
        "Value": "SAM",
        "Key": "lambda:createdBy"
      }
    ]
  }
}

Expected result: InlineCode is uploaded to the lambda function (this is the old behavior as of ~1-2 days ago).

Function:
  Type: AWS::Serverless::Function
  Properties:
    AutoPublishAlias: live
    Handler: index.handler
    Role:
      Fn::GetAtt:
      - Role
      - Arn
    Runtime: nodejs10.x
    InlineCode: "code"
    CodeUri: s3://...

is processed into:

"Function": {
  "Type": "AWS::Lambda::Function",
  "Properties": {
    "Handler": "index.handler",
    "Code": {
      "ZipFile": "code"
    },
    "Role": {
      "Fn::GetAtt": [
        "Role",
        "Arn"
      ]
    },
    "Runtime": "nodejs10.x",
    "Tags": [
      {
        "Value": "SAM",
        "Key": "lambda:createdBy"
      }
    ]
  }
}

The reason I ran into this issue is that aws cloudformation package results in setting CodeUri even when having InlineCode set. This might be due to my specific version of the aws cli. I didn't realize this was the case until starting to look into this issue.

I think ideally this would throw an error rather than defaulting to CodeUri. Another option would be to revert back to the old behavior (where InlineCode takes precedence over CodeUri). Let me know if you need any more information! Thanks!

zsaplan commented 3 years ago

We are running into this same behavior

dehli commented 3 years ago

@zsaplan My workaround was to programmatically remove CodeUri after running aws cloudformation package for the resources that had them both set. This ensured that the code that would be deployed is InlineCode

wchengru commented 3 years ago

Hi @dehli and @zsaplan , can you please let me know how you generate the transformed template? I have tested the template below with SAM translator 1.31.1 and SAM CLI 1.11.0.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Resources:
  Function:
    Type: AWS::Serverless::Function
    Properties:
      AutoPublishAlias: live
      Handler: index.handler
      Role:
        Fn::GetAtt:
        - Role
        - Arn
      Runtime: nodejs10.x
      InlineCode: "code"
      CodeUri: s3://bucket/key

The transformed template is the expected one:

    "Function": {
      "Type": "AWS::Lambda::Function", 
      "Properties": {
        "Handler": "index.handler", 
        "Code": {
          "ZipFile": "code"
        }, 
        "Role": {
          "Fn::GetAtt": [
            "Role", 
            "Arn"
          ]
        }, 
        "Runtime": "nodejs10.x", 
        "Tags": [
          {
            "Value": "SAM", 
            "Key": "lambda:createdBy"
          }
        ]
      }
    }
dehli commented 3 years ago

Thanks @wchengru! I didn't manually generate the transformed template (I let CloudFormation handle that). I am using cdk as well but I don't think that should affect the transforms at all. I determined the processed template by clicking here:

image

This is what my transform is set as (seems to be the same).

AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31

Let me know if there's something else I can surface! Best.

Edit: My SAM template is a nested stack as well. I doubt it has anything to do with the bug but I figured I'd mention it since there were recent changes with nested stacks.

dehli commented 3 years ago

I was able to reproduce the issue with a minimal example. Simply run aws cloudformation deploy with the following template and you'll see that the processed template uses CodeUri rather than InlineCode (you'll need to change CodeUri to a file you have access to).

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Resources:
  Function:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      Runtime: nodejs10.x
      InlineCode: "foo"
      CodeUri: s3://bucket/key

image

image

wchengru commented 3 years ago

Thanks @dehli ! I can reproduce this issue with your example. We will investigate it, and work with cloudformation team on it.

wchengru commented 3 years ago

Hello @dehli, InlineCode and CodeUri shouldn't appear together in the same template. Please remove one of them and try with the correct template.

dehli commented 3 years ago

Thanks! Ya, that's what I've done to fix the issue. I created this mostly because I didn't want it to result in a bug with other people's code. In our case, we were unknowingly setting both keys (seems to be due to aws cloudformation package) and it resulted in some of our lambda functions having incorrect code set for them.

quangvnm commented 3 years ago

I second what @dehli has said, since long time ago (at least half a year ago based on old artifact), aws cloudformation package always force CodeUri property on resource AWS::Serverless::Function even if InlineCode is already defined. This doesn't cause any issue till recently where Transform begins to take preference of CodeUri instead of InlineCode and it breaks every Inline function we have.

If I tried to do the transform manually with bin/sam-translate.py, the output-template is correct though. So maybe sam-translator used by AWS Cloudformation Service is different?

brewneaux commented 3 years ago

Is https://github.com/aws/aws-sam-cli/issues/2388 also caused by this?

jfuss commented 3 years ago

We are rolling back some changes. Please give this about an hour to reflect. Sorry for the impact this might have caused.

@dehli @quangvnm @brewneaux

quangvnm commented 3 years ago

Thank @jfuss , it returns back to normal, even though the CodeUri still exists, I will fire a bug about this on aws-cli repo.

Could you tell us what has been changed and why the sam-translator behavior is different between AWS Cloudformation Service and manual execution of bin/sam-translate.py, please?

dehli commented 3 years ago

Thanks @jfuss! 👍 Also, I found https://github.com/aws/aws-cli/issues/3763 which seems to be the issue with the CLI @quangvnm

sapessi commented 3 years ago

Thank @jfuss , it returns back to normal, even though the CodeUri still exists, I will fire a bug about this on aws-cli repo.

Could you tell us what has been changed and why the sam-translator behavior is different between AWS Cloudformation Service and manual execution of bin/sam-translate.py, please?

We were deploying an internal change that altered the way the SAM translator processes the CodeUri/InlineCode properties. The change caused the CodeUri to override the InlineCode property where both were specified. We have added an integration test to cover this edge case and ensure there are no regressions going forward.

dehli commented 3 years ago

Thanks for the transparency and quick turnaround! Just verified that the processed template is using InlineCode over CodeUri (see attached image). I'll go ahead and close this!

Screen Shot 2020-11-21 at 07 06 58
divyeshramani commented 2 weeks ago

If you deploy with template like below it works

Transform: AWS::Serverless-2016-10-31

Description: Lambda Deployment

Resources: 
  MyFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: python3.11
      MemorySize: 128
      Handler: index.handler
      Policies: AmazonEC2FullAccess
      Timeout: 300
      InlineCode: |
        import boto3

        def handler(event, context):
          print(event)