Closed davidjb closed 9 months ago
Thanks for reporting this issue, I do confirm this is an actual bug, SAM CLI should be validating architecture value earlier. Team will be working on this.
This bug is also permitting Layer builds on incompatible architectures - for example a LayerVersion with CompatibleArchitectures: [x86_64]
is permitted to be built on arm64, resulting in a layer_build_definitions
entry in build.toml
with architecture = "arm64"
in it (assuming that the build doesn't error out on the incompatible platform - e.g. Node.js projects with native code).
After merging #6322 and #6355, and running the same commands mentioned above, we have:
sam build MyLayer
- Still succeeds, as Makefile builds should not fail on validation. However, a warning is now displayed:
WARNING: `fake` in Layer `MyLayer` is not a valid architecture.
sam build --use-container MyLayer
- Does not attempt to download a an image with invalid arch. Instead, fails with the following error:
Error: 'fake' is not a valid architecture, it should be either 'x86_64' or 'arm64'
sam build MyFunction
- Fails as expected with invalid arch
Build Failed
Error: PythonPipBuilder:Validation - Architecture fake is not supported for runtime python3.11
sam build --use-container MyFunction
- Fails as expected with the same message as Layer with --use-container
Error: 'fake' is not a valid architecture, it should be either 'x86_64' or 'arm64'
Parameter and MemorySize validation are not features we plan on implementing with this change set. If you'd like to see these in SAM CLI, please feel free to open up a Feature Request
Patch is released in v1.104.0. Closing
Thanks @bentvelj - really appreciate the breakdown of the changes. I've opened #6510 for MemorySize
and #6511 as an expanded Parameter
validation issue (it seems that no validation occurs on parameters in SAM at all).
Description:
sam build
doesn't attempt to validate values for:Architectures
orMemorySize
CompatibleArchitectures
AllowedValues
Despite the documentation stating that these properties have a set of valid values, the values aren't validated, leading to either successful builds, attempts to download invalid containers, or failure later in the build process. In the case of
AllowedValues
, the effect of this can lead to any sort of breakage, since aParameter
can be used anywhere.This is related to https://github.com/aws/aws-sam-cli/issues/3244 and I wrote in an update to my comment on that issue.
Steps to reproduce:
template.yml
:Shim out
mylayer/Makefile
with:Run builds and observe results, as per next section below.
Observed result:
sam build MyLayer
and observe success (e.g. no validation)sam build --use-container MyLayer
and observe attempt to download a container image from ECR with the invalid arch:Build Failed 2023-10-19 17:13:13,774 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics 2023-10-19 17:13:13,775 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics 2023-10-19 17:13:13,776 | Unable to find Click Context for getting session_id. Error: Could not find public.ecr.aws/sam/build-provided.al2:latest-fake image locally and failed to pull it from docker.
2023-10-19 17:27:40,973 | No config file found in this directory. 2023-10-19 17:27:40,974 | OSError occurred while reading TOML file: [Errno 2] No such file or directory: '/demo/samconfig.toml' 2023-10-19 17:27:40,975 | Config file location: /demo/samconfig.toml 2023-10-19 17:27:40,975 | Config file '/demo/samconfig.toml' does not exist 2023-10-19 17:27:40,977 | Using SAM Template at /demo/template.yml 2023-10-19 17:27:40,993 | OSError occurred while reading TOML file: [Errno 2] No such file or directory: '/demo/samconfig.toml' 2023-10-19 17:27:40,994 | Using config file: samconfig.toml, config environment: default 2023-10-19 17:27:40,994 | Expand command line arguments to: 2023-10-19 17:27:40,995 | --template_file=/demo/template.yml --resource_logical_id=MyFunction --mount_with=READ --build_dir=.aws-sam/build --cache_dir=.aws-sam/cache 2023-10-19 17:27:41,025 | 'build' command is called 2023-10-19 17:27:41,027 | Collected default values for parameters: {'MyParam': 'C'} 2023-10-19 17:27:41,036 | There is no customer defined id or cdk path defined for resource MyLayer, so we will use the resource logical id as the resource id 2023-10-19 17:27:41,037 | There is no customer defined id or cdk path defined for resource MyFunction, so we will use the resource logical id as the resource id 2023-10-19 17:27:41,037 | 0 stacks found in the template 2023-10-19 17:27:41,038 | Collected default values for parameters: {'MyParam': 'C'} 2023-10-19 17:27:41,046 | There is no customer defined id or cdk path defined for resource MyLayer, so we will use the resource logical id as the resource id 2023-10-19 17:27:41,046 | There is no customer defined id or cdk path defined for resource MyFunction, so we will use the resource logical id as the resource id 2023-10-19 17:27:41,047 | 2 resources found in the stack 2023-10-19 17:27:41,047 | Found Serverless function with name='MyFunction' and CodeUri='myfunction' 2023-10-19 17:27:41,047 | --base-dir is not presented, adjusting uri myfunction relative to /demo/template.yml 2023-10-19 17:27:41,048 | --base-dir is not presented, adjusting uri mylayer relative to /demo/template.yml 2023-10-19 17:27:41,049 | 2 resources found in the stack 2023-10-19 17:27:41,050 | Found Serverless function with name='MyFunction' and CodeUri='myfunction' 2023-10-19 17:27:41,050 | Instantiating build definitions 2023-10-19 17:27:41,052 | Same function build definition found, adding function (Previous: BuildDefinition(python3.11, /demo/myfunction, Zip, , 0579b52d-7129-451b-8f4e-b454aec536f3, {}, {}, fake, []), Current: BuildDefinition(python3.11, /demo/myfunction, Zip, , f7e1bfc2-f643-433c-944c-d19eae767730, {}, {}, fake, []), Function: Function(function_id='MyFunction', name='MyFunction', functionname='MyFunction', runtime='python3.11', memory=None, timeout=None, handler='app.lambdaHandler', imageuri=None, packagetype='Zip', imageconfig=None, codeuri='/demo/myfunction', environment=None, rolearn=None, layers=[], events=None, metadata={'SamResourceId': 'MyFunction'}, inlinecode=None, codesign_config_arn=None, architectures=['fake'], function_url_config=None, function_build_info=<FunctionBuildInfo.BuildableZip: ('BuildableZip', 'Regular ZIP function which can be build with SAM CLI')>, stack_path='', runtime_management_config=None)) 2023-10-19 17:27:41,054 | Building codeuri: /demo/myfunction runtime: python3.11 metadata: {} architecture: fake functions: MyFunction 2023-10-19 17:27:41,055 | Building to following folder /demo/.aws-sam/build/MyFunction 2023-10-19 17:27:41,057 | Loading workflow module 'aws_lambda_builders.workflows' 2023-10-19 17:27:41,059 | Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)' 2023-10-19 17:27:41,060 | Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)' 2023-10-19 17:27:41,061 | Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)' 2023-10-19 17:27:41,063 | Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)' 2023-10-19 17:27:41,064 | Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)' 2023-10-19 17:27:41,065 | Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)' 2023-10-19 17:27:41,066 | Registering workflow 'NodejsNpmEsbuildBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm-esbuild', application_framework=None)' 2023-10-19 17:27:41,068 | Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)' 2023-10-19 17:27:41,069 | Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)' 2023-10-19 17:27:41,070 | Registering workflow 'RustCargoLambdaBuilder' with capability 'Capability(language='rust', dependency_manager='cargo', application_framework=None)' 2023-10-19 17:27:41,070 | Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)' 2023-10-19 17:27:41,071 | requirements.txt file not found. Continuing the build without dependencies. 2023-10-19 17:27:41,072 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11 2023-10-19 17:27:41,074 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11 2023-10-19 17:27:41,076 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11 2023-10-19 17:27:41,078 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11 2023-10-19 17:27:41,079 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11 2023-10-19 17:27:41,081 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11 2023-10-19 17:27:41,082 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11 2023-10-19 17:27:41,084 | Runtime validation error for python Traceback (most recent call last): File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflow.py", line 85, in wrapper valid_path = binary_checker.validator.validate(executable_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/workflows/python_pip/validator.py", line 43, in validate runtime_path = super(PythonRuntimeValidator, self).validate(runtime_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sam-installation/lib/python3.11/site-packages/aws_lambda_builders/validator.py", line 74, in validate raise UnsupportedArchitectureError(runtime=self.runtime, architecture=self.architecture) aws_lambda_builders.exceptions.UnsupportedArchitectureError: Architecture fake is not supported for runtime python3.11
Build Failed 2023-10-19 17:27:41,086 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics 2023-10-19 17:27:41,086 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics 2023-10-19 17:27:41,087 | Unable to find Click Context for getting session_id. Error: PythonPipBuilder:Validation - Architecture fake is not supported for runtime python3.11
Starting Build inside a container Building codeuri: /demo/myfunction runtime: python3.11 metadata: {} architecture: fake functions: MyFunction
Build Failed Error: Could not find public.ecr.aws/sam/build-python3.11:latest-fake image locally and failed to pull it from docker.
Expected result:
Validation errors would be shown when the SAM config is loaded, before the values are used for building and builds would not succeed when invalid values are used.
As per https://github.com/aws/aws-sam-cli/issues/3244, some of these values will be validated but only at deploy time, assuming you're able to create a configuration that is able to be built but not deploy (e.g. a Parameter used in a non-build manner).
Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
Ref: