search

aws / aws-codebuild-docker-images

Official AWS CodeBuild repository for managed Docker images http://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref.html
Other
1.11k stars 971 forks source link

update the version in docker-compose.yml files #721

Closed lfatty closed 3 months ago

lfatty commented 3 months ago

Describe the bug The current version that the docker-compose.yml files use need to be update to version: 3.x in for codebuild.sh to run using the latest version of docker desktop

To Reproduce

docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock --entrypoint /bin/sh public.ecr.aws/codebuild/local-builds:latest -c 'docker version' WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested Client: Docker Engine - Community Version: 18.09.0 API version: 1.39 Go version: go1.10.4 Git commit: 4d60db4 Built: Wed Nov 7 00:46:51 2018 OS/Arch: linux/amd64 Experimental: false Server: Docker Desktop 4.31.0 (149459) Engine: Version: 26.1.0 API version: 1.45 (minimum version 1.24) Go version: go1.21.9 Git commit: c8af8eb Built: Mon Apr 22 17:06:59 2024 OS/Arch: linux/arm64 Experimental: false containerd: Version: 1.6.31 GitCommit: e377cd56a71523140ca6ae87e30244719194a521 runc: Version: 1.1.12 GitCommit: v1.1.12-0-g51d5e94 docker-init: Version: 0.19.0 GitCommit: de40ad0

Docker-compose with a “v3” compose-file works;

cat > docker-compose.yml <<'EOF' version: "3.3" services: web: image: nginx:alpine EOF docker-compose up WARNING: The Docker Engine you're using is running in swarm mode. Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node. To deploy your application across the swarm, use docker stack deploy. Creating network "localbuild_default" with the default driver Creating localbuild_web_1 ... done Attaching to localbuild_web_1 web_1 | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration ...

Attempting with with a “v2" compose file makes it use an old API version and fails;

cat > docker-compose.yml <<'EOF' version: "2" services: web: image: nginx:alpine EOF docker-compose up ERROR: client version 1.22 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version

And it looks like the entrypoint script by default uses some compose files that are in the image and are older using version: '2' (which is 2.0 so downgrades API version); in /Localbuild/agent-resources/

docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock --entrypoint /bin/sh public.ecr.aws/codebuild/local-builds:latest -c 'ls -l /LocalBuild/agent-resources/' WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested total 20 drwxr-xr-x 2 root root 4096 Feb 10 2023 bin -rw-r--r-- 1 root root 2340 Jan 13 2023 docker-compose-mount-src-dir.yml -rw-r--r-- 1 root root 2422 Jan 13 2023 docker-compose.yml -rw-r--r-- 1 root root 368 Jan 13 2023 local-build-config.yml -rwxr-xr-x 1 root root 847 Jan 13 2023 start 2 files

Expected behavior ERROR: client version 1.22 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version

Logs N/A

Platform (please complete the following information): Mac Os

Additional context

Server: Docker Desktop 4.30.0 (149282) Engine: Version: 26.1.1 API version: 1.45 (minimum version 1.24) Go version: go1.21.9 Git commit: ac2de55 Built: Tue Apr 30 11:48:28 2024 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.6.31 GitCommit: e377cd56a7

lfatty commented 3 months ago

This file https://github.com/aws/aws-codebuild-docker-images/blob/master/local_builds/codebuild_build.sh uses docker-compose.yml files that are on version 2 which need to be updated to 3.x.

lfatty commented 3 months ago

version: '2'

services: agent: image: ${LOCAL_AGENT_IMAGE} ports:

  • "3000" volumes:
  • source_volume:/codebuild/input
  • user_volume:/codebuild/output
  • ${CODEBUILD_LOCAL_SOURCE_DIRECTORY}:/codebuild/local/src:ro # Mount this to pass the build data yml file which local-CTS will use to return build info. environment:
  • CODEBUILD_LOCAL_BUILD=true # Go-agent attempts to connect to local CTS
  • AWS_ACCESS_KEY_ID=key # Prevent going to EC2 instance metadata.
  • AWS_SECRET_ACCESS_KEY=secret # Prevent going to EC2 instance metadata.
  • CODEBUILD_LOCAL_ECS_AGENT_ACCESS_KEY=
  • CODEBUILD_LOCAL_ECS_AGENT_SECRET_KEY=
  • CODEBUILD_LOCAL_ECS_AGENT_SESSION_TOKEN=
  • CODEBUILD_CONTAINER_TOKENS=12345
  • AWS_CONTAINER_CREDENTIALS_RELATIVE_URI=/credentials
  • CODEBUILD_REGION=us-west-2 # Provides region to use for CTS connection in BMR
  • CODEBUILD_CTS_ENDPOINT=http://localhost:8100
  • CODEBUILD_LOCAL_ECS_AGENT=true
  • CODEBUILD_GOLANG_AGENT=true
  • CODEBUILD_LOCAL_BUILD_DATA_FILE=/LocalBuild/agent-resources/local-build-config.yml
  • CODEBUILD_MOUNT_SRC_DIR=${CODEBUILD_MOUNT_SRC_DIR}
  • BUILDSPEC_PATH=${CODEBUILD_LOCAL_BUILDSPEC_PATH}
  • SOURCE_PATH=${CODEBUILD_LOCAL_SOURCE_DIRECTORY}
  • IS_INNER_CONTAINER=true build: image: ${IMAGE_FOR_CODEBUILD_LOCAL_BUILD} privileged: ${BUILD_CONTAINER_PRIVILEGED_MODE} user: root entrypoint: sh -c "while [ ! -f /codebuild/readonly/bin/executor.done ]; do sleep 1; done && /codebuild/readonly/bin/executor > /dev/null" volumes:
  • source_volume:/codebuild/readonly
  • user_volume:/codebuild/output
  • /var/run/docker.sock:/var/run/docker.sock
  • ${CODEBUILD_LOCAL_ARTIFACTS_DIRECTORY}:/codebuild/output/artifacts environment:
  • NO_PROXY=BMR:3000
  • CODEBUILD_LOCAL_BUILD=true
  • CODEBUILD_AGENT_PORT=http://BMR:3000
  • CODEBUILD_AUTH_TOKEN=12345
  • CODEBUILD_BUILD_ID=local:00000000-0000-0000-0000-000000000000
  • CODEBUILD_BUILD_IMAGE=${IMAGE_FOR_CODEBUILD_LOCAL_BUILD}
  • CODEBUILD_INITIATOR=${CODEBUILD_LOCAL_INITIATOR} links:
  • "agent:BMR" volumes: source_volume: driver: local user_volume: driver: local
lfatty commented 3 months ago

version: '2'

services: agent: image: ${LOCAL_AGENT_IMAGE} ports:

  • "3000" volumes:
  • source_volume:/codebuild/input
  • user_volume:/codebuild/output environment:
  • CODEBUILD_LOCAL_BUILD=true # Go-agent attempts to connect to local CTS
  • AWS_ACCESS_KEY_ID=key # Prevent going to EC2 instance metadata.
  • AWS_SECRET_ACCESS_KEY=secret # Prevent going to EC2 instance metadata.
  • CODEBUILD_LOCAL_ECS_AGENT_ACCESS_KEY=
  • CODEBUILD_LOCAL_ECS_AGENT_SECRET_KEY=
  • CODEBUILD_LOCAL_ECS_AGENT_SESSION_TOKEN=
  • CODEBUILD_CONTAINER_TOKENS=12345
  • AWS_CONTAINER_CREDENTIALS_RELATIVE_URI=/credentials
  • CODEBUILD_REGION=us-west-2 # Provides region to use for CTS connection in BMR
  • CODEBUILD_CTS_ENDPOINT=http://localhost:8100
  • CODEBUILD_LOCAL_ECS_AGENT=true
  • CODEBUILD_GOLANG_AGENT=true
  • CODEBUILD_LOCAL_BUILD_DATA_FILE=/LocalBuild/agent-resources/local-build-config.yml
  • CODEBUILD_MOUNT_SRC_DIR=${CODEBUILD_MOUNT_SRC_DIR}
  • BUILDSPEC_PATH=${CODEBUILD_LOCAL_BUILDSPEC_PATH}
  • SOURCE_PATH=${CODEBUILD_LOCAL_SOURCE_DIRECTORY}
  • IS_INNER_CONTAINER=true build: image: ${IMAGE_FOR_CODEBUILD_LOCAL_BUILD} privileged: ${BUILD_CONTAINER_PRIVILEGED_MODE} user: root entrypoint: sh -c "while [ ! -f /codebuild/readonly/bin/executor.done ]; do sleep 1; done && /codebuild/readonly/bin/executor > /dev/null" volumes:
  • source_volume:/codebuild/readonly
  • user_volume:/codebuild/output
  • /var/run/docker.sock:/var/run/docker.sock
  • ${CODEBUILD_LOCAL_SOURCE_DIRECTORY}:/codebuild/output/srcDownload/src
  • ${CODEBUILD_LOCAL_ARTIFACTS_DIRECTORY}:/codebuild/output/artifacts environment:
  • NO_PROXY=BMR:3000
  • CODEBUILD_LOCAL_BUILD=true
  • CODEBUILD_AGENT_PORT=http://BMR:3000
  • CODEBUILD_AUTH_TOKEN=12345
  • CODEBUILD_BUILD_ID=local:00000000-0000-0000-0000-000000000000
  • CODEBUILD_BUILD_IMAGE=${IMAGE_FOR_CODEBUILD_LOCAL_BUILD}
  • CODEBUILD_INITIATOR=${CODEBUILD_LOCAL_INITIATOR} links:
  • "agent:BMR" volumes: source_volume: driver: local user_volume: driver: local
gregor-mindflow commented 3 months ago

As a workaround you can do this in the meantime : https://stackoverflow.com/a/78406530

lfatty commented 3 months ago

It works for me! Thank you @gregor-mindflow I believe AWS should update the compose file to version 3 so that we can avoid this workaround.