search

aws / aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
https://aws.amazon.com/serverless/sam/
Apache License 2.0
6.52k stars 1.17k forks source link

Bug: SAM build ignores SkipBuild metadata flag when specifying the name of lambda #7507

Open cj-mantas-gridinas opened 1 month ago

cj-mantas-gridinas commented 1 month ago

Description:

Setting the skip build flag only affects lambda builds if you do not specify which lambda should be built using the build subcommand.

Steps to reproduce:

  1. Set Resources.lambda.Metadata.SkipBuild to True
  2. Run sam build --template template.yml lambda

Example template.yml file

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: An AWS Serverless Application Model template describing your function.
Resources:
  lambda:
    Type: AWS::Serverless::Function
    Metadata:
      SkipBuild: True
    Properties:
      FunctionName: "foo"
      CodeUri: .
      Description: lambda with zip containing libraries as dedicated jars
      MemorySize: 2048
      Timeout: 600
      Handler: foo.Main::handleRequest
      Runtime: java21
      Architectures:
        - x86_64
      EphemeralStorage:
        Size: 1024
      EventInvokeConfig:
        MaximumEventAgeInSeconds: 21600
        MaximumRetryAttempts: 2
        DestinationConfig: {}
      PackageType: Zip
      SnapStart:
        ApplyOn: None
      RuntimeManagementConfig:
        UpdateRuntimeOn: Auto
      VpcConfig:
        SecurityGroupIds:
          - sg-1
        SubnetIds:
          - subnet-1
          - subnet-2
        Ipv6AllowedForDualStack: false

Used the following build.gradle to show as proof that the build is still run.

plugins {
    id 'java'
}

group = 'org.example'
version = '1.0-SNAPSHOT'

java {
    sourceCompatibility = JavaVersion.VERSION_21
    targetCompatibility = JavaVersion.VERSION_21
}
repositories {
    mavenCentral()
}

dependencies {
    implementation "shit"
}

test {
    useJUnitPlatform()
}

task buildZip(type: Zip) {
    into('lib') {
        from(jar)
        from(configurations.runtimeClasspath)
    }
}

Observed result:

2024-09-24 13:33:33,417 | No config file found in this directory.                                                                                                                                            
2024-09-24 13:33:33,419 | OSError occurred while reading TOML file: [Errno 2] No such file or directory: '/Users/mantas.gridinas/IdeaProjects/lambda-at-home/samconfig.toml'                                 
2024-09-24 13:33:33,420 | Config file location: /Users/mantas.gridinas/IdeaProjects/lambda-at-home/samconfig.toml                                                                                            
2024-09-24 13:33:33,420 | Config file '/Users/mantas.gridinas/IdeaProjects/lambda-at-home/samconfig.toml' does not exist                                                                                     
2024-09-24 13:33:33,435 | OSError occurred while reading TOML file: [Errno 2] No such file or directory: '/Users/mantas.gridinas/IdeaProjects/lambda-at-home/samconfig.toml'                                 
2024-09-24 13:33:33,436 | Using config file: samconfig.toml, config environment: default                                                                                                                     
2024-09-24 13:33:33,436 | Expand command line arguments to:                                                                                                                                                  
2024-09-24 13:33:33,437 | --template_file=/Users/mantas.gridinas/IdeaProjects/lambda-at-home/template.yml --resource_logical_id=lambda --mount_with=READ --build_dir=.aws-sam/build                          
--cache_dir=.aws-sam/cache                                                                                                                                                                                   
2024-09-24 13:33:33,470 | 'build' command is called                                                                                                                                                          
2024-09-24 13:33:33,484 | No Parameters detected in the template                                                                                                                                             
2024-09-24 13:33:33,500 | There is no customer defined id or cdk path defined for resource lambda, so we will use the resource logical id as the resource id                                                 
2024-09-24 13:33:33,501 | 0 stacks found in the template                                                                                                                                                     
2024-09-24 13:33:33,501 | No Parameters detected in the template                                                                                                                                             
2024-09-24 13:33:33,513 | There is no customer defined id or cdk path defined for resource lambda, so we will use the resource logical id as the resource id                                                 
2024-09-24 13:33:33,514 | 1 resources found in the stack                                                                                                                                                     
2024-09-24 13:33:33,514 | Found Serverless function with name='lambda' and CodeUri='.'                                                                                                                       
2024-09-24 13:33:33,515 | --base-dir is not presented, adjusting uri . relative to /Users/mantas.gridinas/IdeaProjects/lambda-at-home/template.yml                                                           
2024-09-24 13:33:33,515 | Skip building pre-built function: lambda                                                                                                                                           
2024-09-24 13:33:33,518 | 1 resources found in the stack                                                                                                                                                     
2024-09-24 13:33:33,519 | Found Serverless function with name='lambda' and CodeUri='.'                                                                                                                       
2024-09-24 13:33:33,519 | Skip building pre-built function: lambda                                                                                                                                           
2024-09-24 13:33:33,520 | Instantiating build definitions                                                                                                                                                    
2024-09-24 13:33:33,520 | No previous build graph found, generating new one                                                                                                                                  
2024-09-24 13:33:33,521 | Unique function build definition found, adding as new (Function Build Definition: BuildDefinition(java21, /Users/mantas.gridinas/IdeaProjects/lambda-at-home, Zip, ,               
f30b6d8f-7c7b-4484-8908-bc74860671b0, {'SkipBuild': True}, {}, x86_64, []), Function: Function(function_id='lambda', name='lambda', functionname='foo', runtime='java21', memory=2048, timeout=600,          
handler='foo.Main::handleRequest', imageuri=None, packagetype='Zip', imageconfig=None, codeuri='/Users/mantas.gridinas/IdeaProjects/lambda-at-home', environment=None, rolearn=None, layers=[], events=None, 
metadata={'SkipBuild': True, 'SamResourceId': 'lambda'}, inlinecode=None, codesign_config_arn=None, architectures=['x86_64'], function_url_config=None, function_build_info=<FunctionBuildInfo.SkipBuild:    
('SkipBuild', 'A Function which is denoted with SkipBuild in metadata, non buildable')>, stack_path='', runtime_management_config={'UpdateRuntimeOn': 'Auto'}, logging_config=None))                         
2024-09-24 13:33:33,523 | Building codeuri: /Users/mantas.gridinas/IdeaProjects/lambda-at-home runtime: java21 metadata: {'SkipBuild': True} architecture: x86_64 functions: lambda                          
2024-09-24 13:33:33,525 | Building to following folder /Users/mantas.gridinas/IdeaProjects/lambda-at-home/.aws-sam/build/lambda                                                                              
2024-09-24 13:33:33,525 | Looking for a supported build workflow in following directories: ['/Users/mantas.gridinas/IdeaProjects/lambda-at-home', '/Users/mantas.gridinas/IdeaProjects/lambda-at-home']      
2024-09-24 13:33:33,527 | Loading workflow module 'aws_lambda_builders.workflows'                                                                                                                            
2024-09-24 13:33:33,529 | Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'                                    
2024-09-24 13:33:33,530 | Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'                       
2024-09-24 13:33:33,532 | Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'                                      
2024-09-24 13:33:33,534 | Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'                                   
2024-09-24 13:33:33,535 | Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'                                     
2024-09-24 13:33:33,537 | Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'                                      
2024-09-24 13:33:33,539 | Registering workflow 'NodejsNpmEsbuildBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm-esbuild', application_framework=None)'                       
2024-09-24 13:33:33,541 | Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'                                      
2024-09-24 13:33:33,543 | Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'                                  
2024-09-24 13:33:33,546 | Registering workflow 'RustCargoLambdaBuilder' with capability 'Capability(language='rust', dependency_manager='cargo', application_framework=None)'                                
2024-09-24 13:33:33,546 | Found workflow 'JavaGradleWorkflow' to support capabilities 'Capability(language='java', dependency_manager='gradle', application_framework=None)'                                 
2024-09-24 13:33:33,936 | Running workflow 'JavaGradleWorkflow'                                                                                                                                              
2024-09-24 13:33:33,937 |  Running JavaGradleWorkflow:GradleBuild                                                                                                                                            
2024-09-24 13:33:34,937 | JavaGradleWorkflow:GradleBuild failed                                                                                                                                              
Traceback (most recent call last):                                                                                                                                                                           
  File "aws_lambda_builders/workflows/java_gradle/actions.py", line 50, in _build_project                                                                                                                    
  File "aws_lambda_builders/workflows/java_gradle/gradle.py", line 45, in build                                                                                                                              
aws_lambda_builders.workflows.java_gradle.gradle.GradleExecutionError: Gradle Failed: FAILURE: Build failed with an exception.                                                                               

* Where:                                                                                                                                                                                                     
Build file '/Users/mantas.gridinas/IdeaProjects/lambda-at-home/build.gradle' line: 17                                                                                                                        

* What went wrong:                                                                                                                                                                                           
A problem occurred evaluating root project 'lambda-at-home'.                                                                                                                                                 
> Supplied String module notation 'shit' is invalid. Example notations: 'org.gradle:gradle-core:2.2', 'org.mockito:mockito-core:1.9.5:javadoc'.                                                              

* Try:                                                                                                                                                                                                       
> Run with --stacktrace option to get the stack trace.                                                                                                                                                       
> Run with --info or --debug option to get more log output.                                                                                                                                                  
> Run with --scan to get full insights.                                                                                                                                                                      
> Get more help at https://help.gradle.org.                                                                                                                                                                  

BUILD FAILED in 515ms                                                                                                                                                                                        

During handling of the above exception, another exception occurred:                                                                                                                                          

Traceback (most recent call last):                                                                                                                                                                           
  File "aws_lambda_builders/workflow.py", line 374, in run                                                                                                                                                   
  File "aws_lambda_builders/workflows/java_gradle/actions.py", line 32, in execute                                                                                                                           
  File "aws_lambda_builders/workflows/java_gradle/actions.py", line 58, in _build_project                                                                                                                    
aws_lambda_builders.actions.ActionFailedError: Gradle Failed: FAILURE: Build failed with an exception.                                                                                                       

* Where:                                                                                                                                                                                                     
Build file '/Users/mantas.gridinas/IdeaProjects/lambda-at-home/build.gradle' line: 17                                                                                                                        

* What went wrong:                                                                                                                                                                                           
A problem occurred evaluating root project 'lambda-at-home'.                                                                                                                                                 
> Supplied String module notation 'shit' is invalid. Example notations: 'org.gradle:gradle-core:2.2', 'org.mockito:mockito-core:1.9.5:javadoc'.                                                              

* Try:                                                                                                                                                                                                       
> Run with --stacktrace option to get the stack trace.                                                                                                                                                       
> Run with --info or --debug option to get more log output.                                                                                                                                                  
> Run with --scan to get full insights.                                                                                                                                                                      
> Get more help at https://help.gradle.org.                                                                                                                                                                  

BUILD FAILED in 515ms                                                                                                                                                                                        

Build Failed
2024-09-24 13:33:34,954 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics                                                                         
2024-09-24 13:33:35,020 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics                                                                         
2024-09-24 13:33:35,021 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': '32e49431-3054-4146-9016-6ce5d4b87225', 'installationId': '2133b365-0639-4ab7-af37-5c89e4d8834b', 'sessionId':        
'128d9d2b-e096-4fd1-8c8c-2196097d9f90', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.8.13', 'samcliVersion': '1.121.0', 'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 
'commandName': 'sam build', 'metricSpecificAttributes': {'projectType': 'CFN', 'gitOrigin': None, 'projectName': '22b9f27c36f488a986348fa8d5bfdd71a578a2b90fcea7803f17fc141049b099', 'initialCommit': None}, 
'duration': 1518, 'exitReason': 'WorkflowFailedError', 'exitCode': 1}}]}                                                                                                                                     
2024-09-24 13:33:35,021 | Unable to find Click Context for getting session_id.                                                                                                                               
2024-09-24 13:33:35,024 | Sending Telemetry: {'metrics': [{'events': {'requestId': '7dc37973-9b87-4bc7-9bcf-636b3100d990', 'installationId': '2133b365-0639-4ab7-af37-5c89e4d8834b', 'sessionId':            
'128d9d2b-e096-4fd1-8c8c-2196097d9f90', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.8.13', 'samcliVersion': '1.121.0', 'commandName': 'sam build', 'metricSpecificAttributes': {'events':    
[{'event_name': 'SamConfigFileExtension', 'event_value': '.toml', 'thread_id': '43449d2fd6aa4a8298339dbdb6cd7e67', 'time_stamp': '2024-09-24 10:33:33.419', 'exception_name': None}, {'event_name':          
'SamConfigFileExtension', 'event_value': '.toml', 'thread_id': 'b4d6074e1e2b482e91e2c2ec5dd4eced', 'time_stamp': '2024-09-24 10:33:33.436', 'exception_name': None}, {'event_name': 'BuildFunctionRuntime',  
'event_value': 'java21', 'thread_id': '30a5f3de097c4b6483656a68d54f4807', 'time_stamp': '2024-09-24 10:33:33.520', 'exception_name': None}, {'event_name': 'BuildWorkflowUsed', 'event_value': 'java-gradle',
'thread_id': '19f3109f4280404aa35031aa6e4c82ff', 'time_stamp': '2024-09-24 10:33:33.526', 'exception_name': None}]}}}]}                                                                                      
2024-09-24 13:33:35,526 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)                                                   
Error: JavaGradleWorkflow:GradleBuild - Gradle Failed: FAILURE: Build failed with an exception.

* Where:
Build file '/Users/mantas.gridinas/IdeaProjects/lambda-at-home/build.gradle' line: 17

* What went wrong:
A problem occurred evaluating root project 'lambda-at-home'.
> Supplied String module notation 'shit' is invalid. Example notations: 'org.gradle:gradle-core:2.2', 'org.mockito:mockito-core:1.9.5:javadoc'.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

BUILD FAILED in 515ms
2024-09-24 13:33:35,560 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)

Expected result:

Build is skipped regardless of naming the lambda or not as that is how the aws toolkit idea plugin works

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

  1. OS: Mac 14.6
  2. sam --version: 1.121
  3. AWS region: irrelevant
{
  "version": "1.121.0",
  "system": {
    "python": "3.8.13",
    "os": "macOS-14.6.1-arm64-arm-64bit"
  },
  "additional_dependencies": {
    "docker_engine": "26.0.0",
    "aws_cdk": "Not available",
    "terraform": "1.8.3"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}
lucashuy commented 1 month ago

This kind of sounds like an edge case we did just never considered: ie. what happens if we skip the build, but also pass in the function explicitly to build.

I just wanted to ask about your use case, since sam build will still create an output template with other resources when specifying an explicit resource to build. It just won't build the other resources.

cj-mantas-gridinas commented 1 month ago

My usecase is to run lambdas via intellij with aws toolkit plugin. The plugin runs 2 commands sequentially - build, with explicit lambda name, and local invoke with output template and other resources that would be produced by the build command. The plugin isn't something made by community either. It's provided by AWS (verified, too).

https://plugins.jetbrains.com/plugin/11349-aws-toolkit

The original problem stems from SAM trying to guess what build tool is being used, trying to invoke it, and expecting a partcular file to be present after the build. In the provided case, SAM would expect the result to be a fat jar, instead of a zip of jars, which is a valid deployment in regular lambdas. Just moving the zip file to ARTIFACT_DIR (in makefile configuration) isn't enough, as you must extract it.