search

aws / aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
https://aws.amazon.com/serverless/sam/
Apache License 2.0
6.47k stars 1.16k forks source link

Bug: `sam sync --watch` doesn't work for layers because LogicalResourceId changes on each build #6391

Open jrmalin opened 7 months ago

jrmalin commented 7 months ago

Description:

When using sam sync --watch, layers will not update because a layer's LogicalResourceId changes on each build/upload. Thus, when the layer is "updated" due to the watch, other functions will still be pointed at the original layer LogicalResourceId. I do not know of a way to specify a layer's LogicalResourceId.

Steps to reproduce:

  1. Create lambda function reliant on a layer
  2. Run sam sync --watch
  3. Update layer and wait for redeploy
  4. Note that update does not take effect until the watch is killed and re-run

Observed result:

No update to function's layer output

Expected result:

Update to function's layer output

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

{
  "version": "1.94.0",
  "system": {
    "python": "3.8.13",
    "os": "macOS-13.2.1-arm64-arm-64bit"
  },
  "additional_dependencies": {
    "docker_engine": "20.10.20",
    "aws_cdk": "Not available",
    "terraform": "Not available"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}
jysheng123 commented 7 months ago

Hi, thanks for bringing up the issue. I tried to recreate the error using a simple sam project involving a lambda function and layer, however I encountered no error during this. For the LogicalResourceId, it is supposed to change everytime the layer is changed because each id represents a new version of the layer and it should update the stack to be pointing to the new layer version everytime, which is occurring on my end. Could you give me more details about your project, ideally it would the most beneficial if you could provide a sample project that has the error that occurs when you run sam sync --watch. I also downgraded my SAM CLI version to yours and was unable to recreate the issue.

jrmalin commented 6 months ago

Here is the command I'm running: sam sync --stack-name PROJECT_NAME --watch --parameter-overrides DomainName=api.dev.PROJECT.com StageName=Dev CertificateArn=PROJECT_ARN --no-skip-deploy-sync

On a separate note, is there a way that I can avoid including all of the SSMParameterReadPolicy for every function if I want those values for all of them by default?

Here is what the yaml looks like:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: SAM Template for PROJECT

Parameters:
  DomainName:
    Type: String
  CertificateArn:
    Type: String
  StageName:
    Type: String
Globals:
  Function:
    Timeout: 3
    Tracing: Active
    Layers:
      - !Ref UtilsLayer
  Api:
    TracingEnabled: True
    Cors:
      AllowOrigin: "'*'"
      AllowMethods: "'*'"
      AllowHeaders: "'*'"
      #      AllowCredentials: true
      MaxAge: "'86400'"
Resources:
  EfsLambdaVpc:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: "10.0.0.0/16"
  EfsLambdaSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: "EFS + Lambda on SAM Security Group"
      VpcId: !Ref EfsLambdaVpc
      SecurityGroupEgress:
        - CidrIp: "0.0.0.0/0"
          FromPort: 0
          ToPort: 65535
          IpProtocol: tcp
      SecurityGroupIngress:
        - CidrIp: "0.0.0.0/0"
          FromPort: 0
          ToPort: 65535
          IpProtocol: tcp
  EfsLambdaSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref EfsLambdaVpc
      AvailabilityZone: !Select [ 0, !GetAZs '' ]
      MapPublicIpOnLaunch: false
      CidrBlock: "10.0.0.0/24"
  EfsLambdaSubnetB:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref EfsLambdaVpc
      AvailabilityZone: !Select [ 1, !GetAZs '' ]
      MapPublicIpOnLaunch: false
      CidrBlock: "10.0.1.0/24"
  EfsFileSystem:
    Type: AWS::EFS::FileSystem
  MountTargetA:
    Type: AWS::EFS::MountTarget
    Properties:
      FileSystemId: !Ref EfsFileSystem
      SubnetId: !Ref EfsLambdaSubnetA
      SecurityGroups:
        - !Ref EfsLambdaSecurityGroup
  MountTargetB:
    Type: AWS::EFS::MountTarget
    Properties:
      FileSystemId: !Ref EfsFileSystem
      SubnetId: !Ref EfsLambdaSubnetB
      SecurityGroups:
        - !Ref EfsLambdaSecurityGroup
  AccessPoint:
    Type: AWS::EFS::AccessPoint
    Properties:
      FileSystemId: !Ref EfsFileSystem
      PosixUser:
        Gid: "1000"
        Uid: "1000"
      RootDirectory:
        Path: "/lambda"
        CreationInfo:
          OwnerGid: "1000"
          OwnerUid: "1000"
          Permissions: "755"
  ProjectApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: !Ref StageName
      Auth:
        DefaultAuthorizer: OwnerAuthorizer
        Authorizers:
          # TODO: AdminAuthorizer
          # TODO: It's unclear why, but cacheing the lambda authorizer (ReauthorizeEvery > 0) causes permissioning issues.
          OwnerAuthorizer:
            FunctionArn: !GetAtt AuthOwnerFunction.Arn
            Identity:
              ReauthorizeEvery: 0
          WriteAuthorizer:
            FunctionArn: !GetAtt AuthWriteFunction.Arn
            Identity:
              ReauthorizeEvery: 0
          ReadAuthorizer:
            FunctionArn: !GetAtt AuthReadFunction.Arn
            Identity:
              ReauthorizeEvery: 0
        AddDefaultAuthorizerToCorsPreflight: false
      Domain:
        DomainName: !Ref DomainName
        CertificateArn: !Ref CertificateArn
        Route53:
          HostedZoneId: ZONE_ID
        EndpointConfiguration: REGIONAL
      EndpointConfiguration:
        Type: REGIONAL
  AuthOwnerFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: auth/
      Handler: app.owner_handler
      Runtime: python3.9
      Timeout: 20
      Policies:
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/user
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/password
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/host
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/port
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/name
  AuthWriteFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: auth/
      Handler: app.write_handler
      Runtime: python3.9
      Timeout: 20
      Policies:
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/user
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/password
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/host
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/port
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/name
  AuthReadFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: auth/
      Handler: app.read_handler
      Runtime: python3.9
      Timeout: 20
      Policies:
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/user
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/password
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/host
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/port
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/name
  UtilsLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      ContentUri: utils_layer/
      CompatibleRuntimes:
        - python3.9
    Metadata:
      BuildMethod: python3.9
  HealthCheckFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: health_check/
      Handler: app.lambda_handler
      Runtime: python3.9
      Timeout: 20
      Policies:
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/user
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/password
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/host
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/port
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/name
      Events:
        HttpGetSingleton:
          Type: Api
          Properties:
            RestApiId: !Ref ProjectApi
            Path: /health-check
            Method: get
            Auth:
              Authorizer: NONE
  ActionTypesFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: action_types/
      Handler: app.lambda_handler
      Runtime: python3.9
      Timeout: 20
      Policies:
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/user
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/password
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/host
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/port
        - SSMParameterReadPolicy:
            ParameterName: prod/api/database/name
      Events:
        HttpGetCollection:
          Type: Api
          Properties:
            RestApiId: !Ref ProjectApi
            Path: /action-types
            Method: get
            Auth:
              Authorizer: ReadAuthorizer