search

aws / aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
https://aws.amazon.com/serverless/sam/
Apache License 2.0
6.49k stars 1.17k forks source link

Bug: Builds are permitted with invalid Function `Architectures`/`MemorySize`, Layer `CompatibleArchitectures` and Parameter `AllowedValues` #6110

Closed davidjb closed 9 months ago

davidjb commented 10 months ago

Description:

sam build doesn't attempt to validate values for:

Despite the documentation stating that these properties have a set of valid values, the values aren't validated, leading to either successful builds, attempts to download invalid containers, or failure later in the build process. In the case of AllowedValues, the effect of this can lead to any sort of breakage, since a Parameter can be used anywhere.

This is related to https://github.com/aws/aws-sam-cli/issues/3244 and I wrote in an update to my comment on that issue.

Steps to reproduce:

  1. Create template.yml:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Parameters:
  MyParam:
    Default: C
  AllowedValues: [A, B, C, D]

Resources:
  MyLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      ContentUri: mylayer
      CompatibleRuntimes:
        - provided.al2
      CompatibleArchitectures:
        - fake
        - ${MyParam}
    Metadata:
      BuildMethod: makefile
      BuildArchitecture: fake

  MyFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: myfunction
      Handler: app.lambdaHandler
      Runtime: python3.11
      Architectures:
        - fake
      MemorySize: 1

  1. Shim out mylayer/Makefile with:

    build-MyLayer:
echo 'Done'

  2. Run builds and observe results, as per next section below.

Observed result:

  1. Run sam build MyLayer and observe success (e.g. no validation)
  2. Run sam build --use-container MyLayer and observe attempt to download a container image from ECR with the invalid arch: 
    
2023-10-19 17:13:09,717 | No config file found in this directory.
2023-10-19 17:13:09,718 | OSError occurred while reading TOML file: [Errno 2] No such file or directory: '/demo/samconfig.toml'
2023-10-19 17:13:09,719 | Config file location: /demo/samconfig.toml
2023-10-19 17:13:09,719 | Config file '/demo/samconfig.toml' does not exist
2023-10-19 17:13:09,721 | Using SAM Template at /demo/template.yml
2023-10-19 17:13:09,737 | OSError occurred while reading TOML file: [Errno 2] No such file or directory: '/demo/samconfig.toml'
2023-10-19 17:13:09,737 | Using config file: samconfig.toml, config environment: default
2023-10-19 17:13:09,738 | Expand command line arguments to:
2023-10-19 17:13:09,738 | --template_file=/demo/template.yml --use_container --resource_logical_id=MyLayer --mount_with=READ
--build_dir=.aws-sam/build --cache_dir=.aws-sam/cache
2023-10-19 17:13:09,771 | 'build' command is called
2023-10-19 17:13:09,771 | Starting Build inside a container
2023-10-19 17:13:09,772 | No Parameters detected in the template
2023-10-19 17:13:09,782 | There is no customer defined id or cdk path defined for resource MyLayer, so we will use the resource logical id as the
resource id
2023-10-19 17:13:09,782 | There is no customer defined id or cdk path defined for resource MyFunction, so we will use the resource logical id as the
resource id
2023-10-19 17:13:09,783 | 0 stacks found in the template
2023-10-19 17:13:09,783 | No Parameters detected in the template
2023-10-19 17:13:09,791 | There is no customer defined id or cdk path defined for resource MyLayer, so we will use the resource logical id as the
resource id
2023-10-19 17:13:09,791 | There is no customer defined id or cdk path defined for resource MyFunction, so we will use the resource logical id as the
resource id
2023-10-19 17:13:09,792 | 2 resources found in the stack
2023-10-19 17:13:09,792 | Found Serverless function with name='MyFunction' and CodeUri='myfunction'
2023-10-19 17:13:09,792 | --base-dir is not presented, adjusting uri myfunction relative to /demo/template.yml
2023-10-19 17:13:09,793 | --base-dir is not presented, adjusting uri mylayer relative to /demo/template.yml
2023-10-19 17:13:09,794 | 2 resources found in the stack
2023-10-19 17:13:09,795 | Found Serverless function with name='MyFunction' and CodeUri='myfunction'
2023-10-19 17:13:09,795 | Instantiating build definitions
2023-10-19 17:13:09,797 | Same Layer build definition found, adding layer (Previous: LayerBuildDefinition(MyLayer, /demo/mylayer, ,
86cfb97a-b49b-449d-92c2-116db1db668f, makefile, ['provided.al2'], fake, {}), Current: LayerBuildDefinition(MyLayer, /demo/mylayer, ,
99131689-c00d-437e-8d63-62e223500a28, makefile, ['provided.al2'], fake, {}), Layer: <samcli.lib.providers.provider.LayerVersion object at 0x106b0f8d0>)
2023-10-19 17:13:09,798 | Building layer 'MyLayer'
2023-10-19 17:13:09,801 | For container layer build, first compatible runtime is chosen as build target for container.
2023-10-19 17:13:13,771 | Failed to download image with name public.ecr.aws/sam/build-provided.al2
2023-10-19 17:13:13,773 | Container was not created. Skipping deletion

Build Failed 2023-10-19 17:13:13,774 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics 2023-10-19 17:13:13,775 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics 2023-10-19 17:13:13,776 | Unable to find Click Context for getting session_id. Error: Could not find public.ecr.aws/sam/build-provided.al2:latest-fake image locally and failed to pull it from docker.

5. `sam build MyFunction` - build is beginning but eventually fails later in validation of the arch not being compatible

2023-10-19 17:27:40,973 | No config file found in this directory. 2023-10-19 17:27:40,974 | OSError occurred while reading TOML file: [Errno 2] No such file or directory: '/demo/samconfig.toml' 2023-10-19 17:27:40,975 | Config file location: /demo/samconfig.toml 2023-10-19 17:27:40,975 | Config file '/demo/samconfig.toml' does not exist 2023-10-19 17:27:40,977 | Using SAM Template at /demo/template.yml 2023-10-19 17:27:40,993 | OSError occurred while reading TOML file: [Errno 2] No such file or directory: '/demo/samconfig.toml' 2023-10-19 17:27:40,994 | Using config file: samconfig.toml, config environment: default 2023-10-19 17:27:40,994 | Expand command line arguments to: 2023-10-19 17:27:40,995 | --template_file=/demo/template.yml --resource_logical_id=MyFunction --mount_with=READ --build_dir=.aws-sam/build --cache_dir=.aws-sam/cache 2023-10-19 17:27:41,025 | 'build' command is called 2023-10-19 17:27:41,027 | Collected default values for parameters: {'MyParam': 'C'} 2023-10-19 17:27:41,036 | There is no customer defined id or cdk path defined for resource MyLayer, so we will use the resource logical id as the resource id 2023-10-19 17:27:41,037 | There is no customer defined id or cdk path defined for resource MyFunction, so we will use the resource logical id as the resource id 2023-10-19 17:27:41,037 | 0 stacks found in the template 2023-10-19 17:27:41,038 | Collected default values for parameters: {'MyParam': 'C'} 2023-10-19 17:27:41,046 | There is no customer defined id or cdk path defined for resource MyLayer, so we will use the resource logical id as the resource id 2023-10-19 17:27:41,046 | There is no customer defined id or cdk path defined for resource MyFunction, so we will use the resource logical id as the resource id 2023-10-19 17:27:41,047 | 2 resources found in the stack 2023-10-19 17:27:41,047 | Found Serverless function with name='MyFunction' and CodeUri='myfunction' 2023-10-19 17:27:41,047 | --base-dir is not presented, adjusting uri myfunction relative to /demo/template.yml 2023-10-19 17:27:41,048 | --base-dir is not presented, adjusting uri mylayer relative to /demo/template.yml 2023-10-19 17:27:41,049 | 2 resources found in the stack 2023-10-19 17:27:41,050 | Found Serverless function with name='MyFunction' and CodeUri='myfunction' 2023-10-19 17:27:41,050 | Instantiating build definitions 2023-10-19 17:27:41,052 | Same function build definition found, adding function (Previous: BuildDefinition(python3.11, /demo/myfunction, Zip, , 0579b52d-7129-451b-8f4e-b454aec536f3, {}, {}, fake, []), Current: BuildDefinition(python3.11, /demo/myfunction, Zip, , f7e1bfc2-f643-433c-944c-d19eae767730, {}, {}, fake, []), Function: Function(function_id='MyFunction', name='MyFunction', functionname='MyFunction', runtime='python3.11', memory=None, timeout=None, handler='app.lambdaHandler', imageuri=None, packagetype='Zip', imageconfig=None, codeuri='/demo/myfunction', environment=None, rolearn=None, layers=[], events=None, metadata={'SamResourceId': 'MyFunction'}, inlinecode=None, codesign_config_arn=None, architectures=['fake'], function_url_config=None, function_build_info=<FunctionBuildInfo.BuildableZip: ('BuildableZip', 'Regular ZIP function which can be build with SAM CLI')>, stack_path='', runtime_management_config=None)) 2023-10-19 17:27:41,054 | Building codeuri: /demo/myfunction runtime: python3.11 metadata: {} architecture: fake functions: MyFunction 2023-10-19 17:27:41,055 | Building to following folder /demo/.aws-sam/build/MyFunction 2023-10-19 17:27:41,057 | Loading workflow module 'aws_lambda_builders.workflows' 2023-10-19 17:27:41,059 | Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)' 2023-10-19 17:27:41,060 | Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)' 2023-10-19 17:27:41,061 | Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)' 2023-10-19 17:27:41,063 | Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)' 2023-10-19 17:27:41,064 | Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)' 2023-10-19 17:27:41,065 | Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)' 2023-10-19 17:27:41,066 | Registering workflow 'NodejsNpmEsbuildBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm-esbuild', application_framework=None)' 2023-10-19 17:27:41,068 | Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)' 2023-10-19 17:27:41,069 | Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)' 2023-10-19 17:27:41,070 | Registering workflow 'RustCargoLambdaBuilder' with capability 'Capability(language='rust', dependency_manager='cargo', application_framework=None)' 2023-10-19 17:27:41,070 | Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)' 2023-10-19 17:27:41,071 | requirements.txt file not found. Continuing the build without dependencies. 2023-10-19 17:27:41,072 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11 2023-10-19 17:27:41,074 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11 2023-10-19 17:27:41,076 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11 2023-10-19 17:27:41,078 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11 2023-10-19 17:27:41,079 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11 2023-10-19 17:27:41,081 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11 2023-10-19 17:27:41,082 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11 2023-10-19 17:27:41,084 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11

Build Failed 2023-10-19 17:27:41,086 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics 2023-10-19 17:27:41,086 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics 2023-10-19 17:27:41,087 | Unable to find Click Context for getting session_id. Error: PythonPipBuilder:Validation - Architecture fake is not supported for runtime python3.11

7. `sam build --use-container MyFunction` - fails in the same way the Layer does above, attempting to download an non-existent container.

Starting Build inside a container Building codeuri: /demo/myfunction runtime: python3.11 metadata: {} architecture: fake functions: MyFunction

Build Failed Error: Could not find public.ecr.aws/sam/build-python3.11:latest-fake image locally and failed to pull it from docker.

8. `sam build --parameter-overrides MyParam=ZZ -- MyLayer` - build succeeds, despite an invalid Parameter value

10. If the above were fixed, then `sam build MyFunction` would succeed, despite `MemorySize: 1` being an invalid value. Trying to invoke `sam local invoke MyFunction` fails because Docker requires a minimum of 6MB of memory allocated:

    ```sh
    Error: 400 Client Error for http+docker://localhost/v1.35/containers/create: Bad Request ("Minimum memory limit allowed is 6MB")
    Traceback:
    ...

Expected result:

Validation errors would be shown when the SAM config is loaded, before the values are used for building and builds would not succeed when invalid values are used.

As per https://github.com/aws/aws-sam-cli/issues/3244, some of these values will be validated but only at deploy time, assuming you're able to create a configuration that is able to be built but not deploy (e.g. a Parameter used in a non-build manner).

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

{
  "version": "1.102.0",
  "system": {
    "python": "3.11.6",
    "os": "Linux-6.4.16-linuxkit-aarch64-with-glibc2.35"
  },
  "additional_dependencies": {
    "docker_engine": "24.0.6",
    "aws_cdk": "Not available",
    "terraform": "Not available"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}

Ref:

mndeveci commented 10 months ago

Thanks for reporting this issue, I do confirm this is an actual bug, SAM CLI should be validating architecture value earlier. Team will be working on this.

davidjb commented 9 months ago

This bug is also permitting Layer builds on incompatible architectures - for example a LayerVersion with CompatibleArchitectures: [x86_64] is permitted to be built on arm64, resulting in a layer_build_definitions entry in build.toml with architecture = "arm64" in it (assuming that the build doesn't error out on the incompatible platform - e.g. Node.js projects with native code).

bentvelj commented 9 months ago

After merging #6322 and #6355, and running the same commands mentioned above, we have:

  1. sam build MyLayer - Still succeeds, as Makefile builds should not fail on validation. However, a warning is now displayed:

    WARNING: `fake` in Layer `MyLayer` is not a valid architecture.

  2. sam build --use-container MyLayer - Does not attempt to download a an image with invalid arch. Instead, fails with the following error:

    Error: 'fake' is not a valid architecture, it should be either 'x86_64' or 'arm64'

  3. sam build MyFunction - Fails as expected with invalid arch

Build Failed
Error: PythonPipBuilder:Validation - Architecture fake is not supported for runtime python3.11
  1. sam build --use-container MyFunction - Fails as expected with the same message as Layer with --use-container
Error: 'fake' is not a valid architecture, it should be either 'x86_64' or 'arm64'

Parameter and MemorySize validation are not features we plan on implementing with this change set. If you'd like to see these in SAM CLI, please feel free to open up a Feature Request

github-actions[bot] commented 9 months ago

Patch is released in v1.104.0. Closing

davidjb commented 8 months ago

Thanks @bentvelj - really appreciate the breakdown of the changes. I've opened #6510 for MemorySize and #6511 as an expanded Parameter validation issue (it seems that no validation occurs on parameters in SAM at all).