search

aws / aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
https://aws.amazon.com/serverless/sam/
Apache License 2.0
6.5k stars 1.17k forks source link

Bug: Unable to build Docker-based Functions with SAM #5875

Open igorlg opened 1 year ago

igorlg commented 1 year ago

Description:

Unable to build/package any Lambda function as a Docker image. I've simply initiated a new SAM application with runtime=python3.11, type=image, template=hello-world (from here) and ran sam build.

This behaviour is happening in any SAM project I have, regardless of runtime language, number of Functions, etc.

Steps to reproduce:

  1. Initiate new SAM project, with the following set of choices:

1 ; 1 ; N ; 17 ; 2 ; N ; N ; sam-app

sam init

[...]
Which template source would you like to use?
[...]
Choice: 1

Choose an AWS Quick Start application template
[...]
Template: 1

Use the most popular runtime and package type? (Python and zip) [y/N]: N

Which runtime would you like to use?
[...]
    17 - python3.11
[...]
Runtime: 17

What package type would you like to use?
    1 - Zip
    2 - Image
Package type: 2
[...]
Would you like to enable X-Ray tracing on the function(s) in your application?  [y/N]: N
[...]
Would you like to enable monitoring using CloudWatch Application Insights? [y/N]: N
[...]
Project name [sam-app]:
[...]
    -----------------------
    Generating application:
    -----------------------
    Name: sam-app
    Base Image: amazon/python3.11-base
    Architectures: x86_64
    Dependency Manager: pip
    Output Directory: .
    Configuration file: sam-app/samconfig.toml
[...]
  1. Build application:
sam build

Observed result:

$ SAM_DEBUG=1 sam build
2023-09-01 10:14:09,537 | Return available internal hook packages
2023-09-01 10:14:09,582 | Using config file: samconfig.toml, config environment: default
2023-09-01 10:14:09,583 | Expand command line arguments to:
2023-09-01 10:14:09,584 | --template_file=/Users/xxx/tmp/sam-app/template.yaml --parallel --mount_with=READ --build_dir=.aws-sam/build --cache_dir=.aws-sam/cache
2023-09-01 10:14:09,652 | 'build' command is called
2023-09-01 10:14:09,659 | No Parameters detected in the template
2023-09-01 10:14:09,682 | There is no customer defined id or cdk path defined for resource HelloWorldFunction, so we will use the resource logical id as the resource id
2023-09-01 10:14:09,683 | There is no customer defined id or cdk path defined for resource ServerlessRestApi, so we will use the resource logical id as the resource id
2023-09-01 10:14:09,685 | 0 stacks found in the template
2023-09-01 10:14:09,686 | No Parameters detected in the template
2023-09-01 10:14:09,703 | There is no customer defined id or cdk path defined for resource HelloWorldFunction, so we will use the resource logical id as the resource id
2023-09-01 10:14:09,704 | There is no customer defined id or cdk path defined for resource ServerlessRestApi, so we will use the resource logical id as the resource id
2023-09-01 10:14:09,705 | 2 resources found in the stack
2023-09-01 10:14:09,706 | Found Serverless function with name='HelloWorldFunction' and ImageUri='None'
2023-09-01 10:14:09,707 | --base-dir is not presented, adjusting uri ./hello_world relative to /Users/xxx/tmp/sam-app/template.yaml
2023-09-01 10:14:09,708 | --base-dir is not presented, adjusting uri . relative to /Users/xxx/tmp/sam-app/template.yaml
2023-09-01 10:14:09,715 | 2 resources found in the stack
2023-09-01 10:14:09,716 | Found Serverless function with name='HelloWorldFunction' and ImageUri='None'
2023-09-01 10:14:09,718 | Error occurred while trying to track an event: Event 'BuildFunctionRuntime' does not accept value 'None'.
2023-09-01 10:14:09,719 | Instantiating build definitions
2023-09-01 10:14:09,723 | Unique function build definition found, adding as new (Function Build Definition: BuildDefinition(None, /Users/xxx/tmp/sam-app, Image, , aee1f313-f5e2-4491-a24b-853620700b46,
{'Dockerfile': 'Dockerfile', 'DockerContext': '/Users/xxx/tmp/sam-app/hello_world', 'DockerTag': 'python3.11-v1'}, {}, x86_64, []), Function: Function(function_id='HelloWorldFunction',
name='HelloWorldFunction', functionname='HelloWorldFunction', runtime=None, memory=None, timeout=3, handler=None, imageuri=None, packagetype='Image', imageconfig=None, codeuri='/Users/xxx/tmp/sam-app',
environment=None, rolearn=None, layers=[], events={'HelloWorld': {'Type': 'Api', 'Properties': {'Path': '/hello', 'Method': 'get', 'RestApiId': 'ServerlessRestApi'}}}, metadata={'Dockerfile': 'Dockerfile', 'DockerContext':
'/Users/xxx/tmp/sam-app/hello_world', 'DockerTag': 'python3.11-v1', 'SamResourceId': 'HelloWorldFunction'}, inlinecode=None, codesign_config_arn=None, architectures=['x86_64'], function_url_config=None,
function_build_info=<FunctionBuildInfo.BuildableImage: ('BuildableImage', 'Regular IMAGE function which can be build with SAM CLI')>, stack_path='', runtime_management_config=None))
2023-09-01 10:14:09,730 | Async execution started
2023-09-01 10:14:09,731 | Invoking function functools.partial(<bound method ParallelBuildStrategy.build_single_function_definition of <samcli.lib.build.build_strategy.ParallelBuildStrategy object at 0x107a2dbb0>>,
<samcli.lib.build.build_graph.FunctionBuildDefinition object at 0x107973b50>)
2023-09-01 10:14:09,733 | Building codeuri: /Users/xxx/tmp/sam-app runtime: None metadata: {'Dockerfile': 'Dockerfile', 'DockerContext':
'/Users/xxx/tmp/sam-app/hello_world', 'DockerTag': 'python3.11-v1'} architecture: x86_64 functions: HelloWorldFunction
2023-09-01 10:14:09,735 | Building to following folder /Users/xxx/tmp/sam-app/.aws-sam/build/HelloWorldFunction
2023-09-01 10:14:09,736 | Waiting for async results
2023-09-01 10:14:09,736 | Building image for HelloWorldFunction function
2023-09-01 10:14:09,745 | Setting DockerBuildArgs: {} for HelloWorldFunction function
2023-09-01 10:14:11,596 | Exception raised during the execution
2023-09-01 10:14:11,598 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2023-09-01 10:14:11,664 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2023-09-01 10:14:11,666 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': '13febf5a-4a5d-4139-be00-11bbcbc3e543', 'installationId': 'e80a91d0-ab45-403a-bc04-b0f2843ea1ac', 'sessionId':
'a229cbb7-a8ff-4544-b6a6-3165187d55b5', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.8.13', 'samcliVersion': '1.96.0', 'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build',
'metricSpecificAttributes': {'projectType': 'CFN', 'gitOrigin': None, 'projectName': '593ab2ca51e925b9f6c2f258bc55ed5926cf6d2c78239a685d65907e4ec7edd3', 'initialCommit': None}, 'duration': 2015, 'exitReason': 'StreamParseError',
'exitCode': 255}}]}
2023-09-01 10:14:11,666 | Unable to find Click Context for getting session_id.
2023-09-01 10:14:11,673 | Sending Telemetry: {'metrics': [{'events': {'requestId': '03878a8e-567d-4aeb-a833-a07261f7a4d1', 'installationId': 'e80a91d0-ab45-403a-bc04-b0f2843ea1ac', 'sessionId': 'a229cbb7-a8ff-4544-b6a6-3165187d55b5',
'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.8.13', 'samcliVersion': '1.96.0', 'commandName': 'sam build', 'metricSpecificAttributes': {'events': [{'event_name': 'SamConfigFileExtension', 'event_value': '.toml',
'thread_id': '1728a1be46e64148b323d363ff24fdd3', 'time_stamp': '2023-09-01 00:14:09.547', 'exception_name': None}]}}}]}
2023-09-01 10:14:12,359 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
2023-09-01 10:14:12,362 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)

Error: Expecting value: line 1 column 1 (char 0)
Traceback:
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1688, in invoke
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
  File "click/decorators.py", line 92, in new_func
  File "click/core.py", line 783, in invoke
  File "samcli/lib/telemetry/metric.py", line 184, in wrapped
  File "samcli/lib/telemetry/metric.py", line 149, in wrapped
  File "samcli/lib/utils/version_checker.py", line 42, in wrapped
  File "samcli/cli/main.py", line 95, in wrapper
  File "samcli/commands/build/command.py", line 170, in cli
  File "samcli/commands/build/command.py", line 266, in do_cli
  File "samcli/commands/build/build_context.py", line 281, in run
  File "samcli/lib/build/app_builder.py", line 217, in build
  File "samcli/lib/build/build_strategy.py", line 394, in build
  File "samcli/lib/build/build_strategy.py", line 81, in build
  File "samcli/lib/build/build_strategy.py", line 400, in _build_functions
  File "samcli/lib/build/build_strategy.py", line 416, in _run_builds_async
  File "samcli/lib/utils/async_utils.py", line 131, in run_async
  File "samcli/lib/utils/async_utils.py", line 90, in run_given_tasks_async
  File "asyncio/base_events.py", line 616, in run_until_complete
  File "samcli/lib/utils/async_utils.py", line 58, in _run_given_tasks_async
  File "concurrent/futures/thread.py", line 57, in run
  File "samcli/lib/build/build_strategy.py", line 427, in build_single_function_definition
  File "samcli/lib/build/build_strategy.py", line 164, in build_single_function_definition
  File "samcli/lib/build/app_builder.py", line 643, in _build_function
  File "samcli/lib/build/app_builder.py", line 419, in _build_lambda_image
  File "docker/models/images.py", line 304, in build
  File "docker/utils/json_stream.py", line 75, in split_buffer

An unexpected error was encountered while executing "sam build".
Search for an existing issue:
https://github.com/aws/aws-sam-cli/issues?q=is%3Aissue+is%3Aopen+Bug%3A%20sam%20build%20-%20StreamParseError
Or create a bug report:
https://github.com/aws/aws-sam-cli/issues/new?template=Bug_report.md&title=Bug%3A%20sam%20build%20-%20StreamParseError

Expected result:

Application should successfully build function image.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

  1. OS: MacOS Ventura 13.5.1
  2. sam --version: 1.96.0
  3. AWS region: ap-southeast-2
  4. env DOCKER_HOST: unix:///Users/xxx/.docker/run/docker.sock
{
  "version": "1.96.0",
  "system": {
    "python": "3.8.13",
    "os": "macOS-13.5.1-x86_64-i386-64bit"
  },
  "additional_dependencies": {
    "docker_engine": "22.06.0-beta.0-926-g914b02ebaf.m",
    "aws_cdk": "Not available",
    "terraform": "Not available"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}
$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.10.4)
  compose: Docker Compose (Docker Inc., v2.17.2)
  dev: Docker Dev Environments (Docker Inc., v0.1.0)
  extension: Manages Docker extensions (Docker Inc., v0.2.19)
  init: Creates Docker-related starter files for your project (Docker Inc., v0.1.0-beta.2)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.25.0)
  scout: Command line tool for Docker Scout (Docker Inc., v0.9.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 3
 Server Version: 22.06.0-beta.0-926-g914b02ebaf.m
 Storage Driver: stargz
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2456e983eb9e37e47538f59ea18f2043c9a73640
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.49-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 6
 Total Memory: 7.675GiB
 Name: docker-desktop
 ID: 3629e7ad-254c-4917-9ff2-1dc1c10321f9
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: true
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false
igorlg commented 1 year ago

Note: I've tried the steps described in #4771 and #5646 , without success.

hawflau commented 1 year ago

Hi @igorlg thank you for raising the issue. I noticed you are using a beta/pre-release version of docker. Could you please try using a release version of docker?