search

aws / aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
https://aws.amazon.com/serverless/sam/
Apache License 2.0
6.5k stars 1.17k forks source link

Bug: sam build multiple lambdas only copies first handler (Golang) #6117

Closed atcherry closed 10 months ago

atcherry commented 11 months ago

Description:

Running sam build with a template that defines multiple functions with the same CodeUri but different handlers results in only the first defined handler being generated for both functions.

Steps to reproduce:

My file structure is the following. The contents of the go files are otherwise empty aside from the main.go files having the main function:

project  /
├── src/
│   ├── lambda/
│   │   ├── function-a/
│   │   │   └── main.go
│   │   └── function-b/
│   │       └── main.go
│   ├── common.go
│   ├── ... other common code
│   ├── go.mod
│   └── go.sum
└── template.yaml

This is the template.yaml I used:

AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-1016-10-31
Resources:
  FunctionA:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: FunctionA
      CodeUri: src
      Handler: lambda/function-a
      Runtime: go1.x
  FunctionB:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: FunctionB
      CodeUri: src
      Handler: lambda/function-b
      Runtime: go1.x

Run sam build --use-container (same output occurs regardless of --use-container)

Observed result:

The generated .aws-sam directory results in two binaries of the first defined function in the template.

.aws-sam/
├── build/
│   ├── FunctionA/
│   │   └── lambda/
│   │       └── function-a
│   ├── FunctionB/
│   │   └── lambda/
│   │       └── function-a
│   └── template.yaml
└── build.toml

Here is the output from sam build --use-container --debug

Output 

2023-10-19 12:26:23,506 | No config file found in this directory.                                                                                                                                                                                                                                                                                                                                                                                                                            
2023-10-19 12:26:23,509 | OSError occurred while reading TOML file: [Errno 2] No such file or directory: '/home/acherry/Documents/dev/playground/samtest/samconfig.toml'                                                                                                                                                                                                                                                                                                                     
2023-10-19 12:26:23,510 | Config file location: /home/acherry/Documents/dev/playground/samtest/samconfig.toml                                                                                                                                                                                                                                                                                                                                                                                
2023-10-19 12:26:23,511 | Config file '/home/acherry/Documents/dev/playground/samtest/samconfig.toml' does not exist                                                                                                                                                                                                                                                                                                                                                                         
2023-10-19 12:26:23,514 | Using SAM Template at /home/acherry/Documents/dev/playground/samtest/template.yaml                                                                                                                                                                                                                                                                                                                                                                                 
2023-10-19 12:26:23,535 | OSError occurred while reading TOML file: [Errno 2] No such file or directory: '/home/acherry/Documents/dev/playground/samtest/samconfig.toml'                                                                                                                                                                                                                                                                                                                     
2023-10-19 12:26:23,536 | Using config file: samconfig.toml, config environment: default                                                                                                                                                                                                                                                                                                                                                                                                     
2023-10-19 12:26:23,537 | Expand command line arguments to:                                                                                                                                                                                                                                                                                                                                                                                                                                  
2023-10-19 12:26:23,538 | --template_file=/home/acherry/Documents/dev/playground/samtest/template.yaml --use_container --mount_with=READ --build_dir=.aws-sam/build --cache_dir=.aws-sam/cache                                                                                                                                                                                                                                                                                               
2023-10-19 12:26:23,613 | 'build' command is called                                                                                                                                                                                                                                                                                                                                                                                                                                          
2023-10-19 12:26:23,614 | Starting Build inside a container                                                                                                                                                                                                                                                                                                                                                                                                                                  
2023-10-19 12:26:23,616 | No Parameters detected in the template                                                                                                                                                                                                                                                                                                                                                                                                                             
2023-10-19 12:26:23,637 | There is no customer defined id or cdk path defined for resource FunctionA, so we will use the resource logical id as the resource id                                                                                                                                                                                                                                                                                                                              
2023-10-19 12:26:23,638 | There is no customer defined id or cdk path defined for resource FunctionB, so we will use the resource logical id as the resource id                                                                                                                                                                                                                                                                                                                              
2023-10-19 12:26:23,639 | 0 stacks found in the template                                                                                                                                                                                                                                                                                                                                                                                                                                     
2023-10-19 12:26:23,640 | No Parameters detected in the template                                                                                                                                                                                                                                                                                                                                                                                                                             
2023-10-19 12:26:23,658 | There is no customer defined id or cdk path defined for resource FunctionA, so we will use the resource logical id as the resource id                                                                                                                                                                                                                                                                                                                              
2023-10-19 12:26:23,659 | There is no customer defined id or cdk path defined for resource FunctionB, so we will use the resource logical id as the resource id                                                                                                                                                                                                                                                                                                                              
2023-10-19 12:26:23,660 | 2 resources found in the stack                                                                                                                                                                                                                                                                                                                                                                                                                                     
2023-10-19 12:26:23,661 | Found Serverless function with name='FunctionA' and CodeUri='src'                                                                                                                                                                                                                                                                                                                                                                                                  
2023-10-19 12:26:23,662 | --base-dir is not presented, adjusting uri src relative to /home/acherry/Documents/dev/playground/samtest/template.yaml                                                                                                                                                                                                                                                                                                                                            
2023-10-19 12:26:23,662 | Found Serverless function with name='FunctionB' and CodeUri='src'                                                                                                                                                                                                                                                                                                                                                                                                  
2023-10-19 12:26:23,663 | --base-dir is not presented, adjusting uri src relative to /home/acherry/Documents/dev/playground/samtest/template.yaml                                                                                                                                                                                                                                                                                                                                            
2023-10-19 12:26:23,666 | 2 resources found in the stack                                                                                                                                                                                                                                                                                                                                                                                                                                     
2023-10-19 12:26:23,666 | Found Serverless function with name='FunctionA' and CodeUri='src'                                                                                                                                                                                                                                                                                                                                                                                                  
2023-10-19 12:26:23,667 | Found Serverless function with name='FunctionB' and CodeUri='src'                                                                                                                                                                                                                                                                                                                                                                                                  
2023-10-19 12:26:23,669 | Instantiating build definitions                                                                                                                                                                                                                                                                                                                                                                                                                                    
2023-10-19 12:26:23,671 | Same function build definition found, adding function (Previous: BuildDefinition(go1.x, /home/acherry/Documents/dev/playground/samtest/src, Zip, , 499d0d0a-b3dd-49a5-9bad-e86707bc91c4, {}, {}, x86_64, []), Current: BuildDefinition(go1.x, /home/acherry/Documents/dev/playground/samtest/src, Zip, , 84a8d2d4-d5d8-440b-aad8-12d291be11ce, {}, {}, x86_64, []), Function: Function(function_id='FunctionA', name='FunctionA', functionname='FunctionA',        
runtime='go1.x', memory=None, timeout=None, handler='lambda/function-a', imageuri=None, packagetype='Zip', imageconfig=None, codeuri='/home/acherry/Documents/dev/playground/samtest/src', environment=None, rolearn=None, layers=[], events=None, metadata={'SamResourceId': 'FunctionA'}, inlinecode=None, codesign_config_arn=None, architectures=None, function_url_config=None, function_build_info=, stack_path='', runtime_management_config=None))                                                                                                                                                                                                                                                                                                                                                                                                                       
2023-10-19 12:26:23,676 | Same function build definition found, adding function (Previous: BuildDefinition(go1.x, /home/acherry/Documents/dev/playground/samtest/src, Zip, , 499d0d0a-b3dd-49a5-9bad-e86707bc91c4, {}, {}, x86_64, ['FunctionA']), Current: BuildDefinition(go1.x, /home/acherry/Documents/dev/playground/samtest/src, Zip, , c25a6ae8-684c-4ca3-9894-7aebc23f1cc0, {}, {}, x86_64, []), Function: Function(function_id='FunctionB', name='FunctionB',                       
functionname='FunctionB', runtime='go1.x', memory=None, timeout=None, handler='lambda/function-b', imageuri=None, packagetype='Zip', imageconfig=None, codeuri='/home/acherry/Documents/dev/playground/samtest/src', environment=None, rolearn=None, layers=[], events=None, metadata={'SamResourceId': 'FunctionB'}, inlinecode=None, codesign_config_arn=None, architectures=None, function_url_config=None, function_build_info=, stack_path='', runtime_management_config=None))                                                                                                                                                                                                                                                                                                                                                                                             
2023-10-19 12:26:23,681 | Building codeuri: /home/acherry/Documents/dev/playground/samtest/src runtime: go1.x metadata: {} architecture: x86_64 functions: FunctionA, FunctionB                                                                                                                                                                                                                                                                                                              
2023-10-19 12:26:23,682 | Building to following folder /home/acherry/Documents/dev/playground/samtest/.aws-sam/build/FunctionA                                                                                                                                                                                                                                                                                                                                                               

Fetching public.ecr.aws/sam/build-go1.x:latest-x86_64 Docker container image......
2023-10-19 12:26:24,122 | Mounting /home/acherry/Documents/dev/playground/samtest/src as /tmp/samcli/source:ro,delegated, inside runtime container                                                                                                                                                                                                                                                                                                                                           
Using the request object from command line argument
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'NodejsNpmEsbuildBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm-esbuild', application_framework=None)'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'RustCargoLambdaBuilder' with capability 'Capability(language='rust', dependency_manager='cargo', application_framework=None)'
Found workflow 'GoModulesBuilder' to support capabilities 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Running workflow 'GoModulesBuilder'
 Running GoModulesBuilder:Build
GoModulesBuilder:Build succeeded
2023-10-19 12:26:24,565 | Build inside container returned response {"jsonrpc": "2.0", "id": 1, "result": {"artifacts_dir": "/tmp/samcli/artifacts"}}                                                                                                                                                                                                                                                                                                                                         
2023-10-19 12:26:24,566 | Build inside container was successful. Copying artifacts from container to host                                                                                                                                                                                                                                                                                                                                                                                    
2023-10-19 12:26:24,918 | Copying from container: /tmp/samcli/artifacts/. -> /home/acherry/Documents/dev/playground/samtest/.aws-sam/build/FunctionA                                                                                                                                                                                                                                                                                                                                         
2023-10-19 12:26:24,943 | Build inside container succeeded                                                                                                                                                                                                                                                                                                                                                                                                                                   
2023-10-19 12:26:24,944 | Copying artifacts from /home/acherry/Documents/dev/playground/samtest/.aws-sam/build/FunctionA to /home/acherry/Documents/dev/playground/samtest/.aws-sam/build/FunctionB                                                                                                                                                                                                                                                                                          
2023-10-19 12:26:24,945 | There is no customer defined id or cdk path defined for resource FunctionA, so we will use the resource logical id as the resource id                                                                                                                                                                                                                                                                                                                              
2023-10-19 12:26:24,946 | There is no customer defined id or cdk path defined for resource FunctionB, so we will use the resource logical id as the resource id                                                                                                                                                                                                                                                                                                                              
2023-10-19 12:26:24,947 | 2 resources found in the stack                                                                                                                                                                                                                                                                                                                                                                                                                                     
2023-10-19 12:26:24,947 | Found Serverless function with name='FunctionA' and CodeUri='src'                                                                                                                                                                                                                                                                                                                                                                                                  
2023-10-19 12:26:24,948 | Found Serverless function with name='FunctionB' and CodeUri='src'                                                                                                                                                                                                                                                                                                                                                                                                  

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Expected result:

I would have expected .aws-sam/build/FunctionB/lambda/ to contain function-b and not function-a since that is what the handler is defined as in the template. The debug output even shows that the binary for FunctionB is being coppied from FunctionA.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

{
  "version": "1.99.0",
  "system": {
    "python": "3.11.3",
    "os": "Linux-6.2.0-34-generic-x86_64-with-glibc2.37"
  },
  "additional_dependencies": {
    "docker_engine": "24.0.6",
    "aws_cdk": "Not available",
    "terraform": "1.6.1"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}
atcherry commented 11 months ago

I was able to work around this by setting the build method to makefile with a makefile that does some simple go building.

mndeveci commented 11 months ago

Thanks for reporting the issue.

I was able to re-produce it on my side. AFAIR this should be fixed earlier (see: https://github.com/aws/aws-sam-cli/issues/3894)

Will provide more updates after checking the root cause.

Update: Found the root cause of the issue, we will be working on a fix soon. As a workaround for now, you can add metadata for both functions so that they will have separate build flow. If you update your original template to this, it should work.

AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-1016-10-31
Resources:
  FunctionA:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: FunctionA
      CodeUri: src
      Handler: lambda/function-a
      Runtime: go1.x
    Metadata:
      FunctionName: FunctionA

  FunctionB:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: FunctionB
      CodeUri: src
      Handler: lambda/function-b
      Runtime: go1.x
    Metadata:
      FunctionName: FunctionB
sidhujus commented 10 months ago

Fixed in #6202 and will be rolled out soon

github-actions[bot] commented 10 months ago

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

github-actions[bot] commented 10 months ago

Patch is released in v1.101.0. Closing

atcherry commented 10 months ago

@sidhujus I am not using provided.al2 , but I am curious if these changes cope with that runtime. I should probably test this myself, but I figured I'd at least throw this thought out there as I am not using the provided.al2 runtime and do not know much about it. I just know that the go1.x runtime will be phased out according to this article

https://aws.amazon.com/blogs/compute/migrating-aws-lambda-functions-from-the-go1-x-runtime-to-the-custom-runtime-on-amazon-linux-2/