Closed JensRantil closed 5 months ago
Would need logs from your workflow and/or link to your repo to help you out.
I'm having the same problem!
solved by passing in load: true
this should be documented, because both imageid
and digest
outputs show up as empty, and the metadata
json is incomplete
Currently having the same issue. In my case, we can't use load
and output
together.
name: Publish Image (8.2)
on:
push:
branches: [main]
paths:
- 8.2/**
env:
REGISTRY_IMAGE: public.ecr.aws/luminsports/php-runtime
jobs:
build:
name: Build
runs-on: ${{ matrix.runs-on }}
strategy:
fail-fast: false
matrix:
platform: [linux/amd64, linux/arm64]
include:
- platform: linux/amd64
runs-on: [self-hosted, linux, x64]
- platform: linux/arm64
runs-on: [self-hosted, linux, arm64]
steps:
- name: Check out the repo
uses: actions/checkout@v3
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
with:
images: ${{ env.REGISTRY_IMAGE }}
tags: |
type=ref,event=branch,prefix=8.2-
type=sha,prefix=8.2-
type=raw,value=8.2
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Login to Public ECR
uses: docker/login-action@v2
with:
registry: public.ecr.aws
username: ${{ secrets.AWS_ACCESS_KEY_ID }}
password: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
env:
AWS_REGION: 'us-east-1'
- name: Build and push by digest
id: build
uses: docker/build-push-action@v4
with:
context: .
file: './8.2/Dockerfile'
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ env.REGISTRY_IMAGE }},push-by-digest=true,name-canonical=true,push=true
- name: Export digest
run: |
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@v3
with:
name: digests
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
merge:
name: Merge and Push image manifest
runs-on: ubuntu-latest
needs:
- build
steps:
- name: Download digests
uses: actions/download-artifact@v3
with:
name: digests
path: /tmp/digests
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Docker meta
id: meta
uses: docker/metadata-action@v4
with:
tags: |
type=ref,event=branch,prefix=8.2-
type=sha,prefix=8.2-
type=raw,value=8.2
images: ${{ env.REGISTRY_IMAGE }}
- name: Login to Public ECR
uses: docker/login-action@v2
with:
registry: public.ecr.aws
username: ${{ secrets.AWS_ACCESS_KEY_ID }}
password: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
env:
AWS_REGION: 'us-east-1'
- name: Create manifest list and push
working-directory: /tmp/digests
run: |
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ env.REGISTRY_IMAGE }}@sha256:%s ' *)
- name: Inspect image
run: |
docker buildx imagetools inspect ${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }}
Basically a copy-paste of what is documented, except I have added tags, and use AWS ECR as a registry.
Run docker/setup-buildx-action@v2
with:
driver: docker-container
buildkitd-flags: --allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host
install: false
use: true
cleanup: true
env:
REGISTRY_IMAGE: public.ecr.aws/luminsports/php-runtime
DOCKER_METADATA_OUTPUT_VERSION: 8.2-main
DOCKER_METADATA_OUTPUT_TAGS: public.ecr.aws/luminsports/php-runtime:8.2-main
public.ecr.aws/luminsports/php-runtime:8.2
public.ecr.aws/luminsports/php-runtime:8.2-55ee41e
DOCKER_METADATA_OUTPUT_LABELS: org.opencontainers.image.created=2023-08-01T07:03:50.110Z
org.opencontainers.image.description=
org.opencontainers.image.licenses=
org.opencontainers.image.revision=55ee41e49966302b5739eb0a054d6d6698d03316
org.opencontainers.image.source=https://github.com/luminsports/docker-php-runtime
org.opencontainers.image.title=docker-php-runtime
org.opencontainers.image.url=https://github.com/luminsports/docker-php-runtime
org.opencontainers.image.version=8.2-main
DOCKER_METADATA_OUTPUT_JSON: {"tags":["public.ecr.aws/luminsports/php-runtime:8.2-main","public.ecr.aws/luminsports/php-runtime:8.2","public.ecr.aws/luminsports/php-runtime:8.2-55ee41e"],"labels":{"org.opencontainers.image.created":"2023-08-01T07:03:50.110Z","org.opencontainers.image.description":"","org.opencontainers.image.licenses":"","org.opencontainers.image.revision":"55ee41e49966302b5739eb0a054d6d6698d03316","org.opencontainers.image.source":"https://github.com/luminsports/docker-php-runtime","org.opencontainers.image.title":"docker-php-runtime","org.opencontainers.image.url":"https://github.com/luminsports/docker-php-runtime","org.opencontainers.image.version":"8.2-main"}}
DOCKER_METADATA_OUTPUT_BAKE_FILE: /tmp/docker-actions-toolkit-KZi5wP/docker-metadata-action-bake.json
Docker info
/bin/docker version
Client:
Version: 20.10.23
API version: 1.41
Go version: go1.18.9
Git commit: 7155243
Built: Tue Apr 11 22:56:36 2023
OS/Arch: linux/amd64
Context: default
Experimental: true
Server:
Engine:
Version: 20.10.23
API version: 1.41 (minimum version 1.12)
Go version: go1.18.9
Git commit: 6051f14
Built: Tue Apr 11 22:57:17 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.19
GitCommit: 1e1ea6e986c6c86565bc33d52e34b81b3e2bc71f
runc:
Version: 1.1.7
GitCommit: f19387a6bec4944c770f7668ab51c4348d9c2f38
docker-init:
Version: 0.19.0
GitCommit: de40ad0
/bin/docker info
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc., 0.0.0+unknown)
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 2
Server Version: 20.10.23
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
Default Runtime: runc
Init Binary: docker-init
containerd version: 1e1ea6e986c6c86565bc33d52e34b81b3e2bc71f
runc version: f19387a6bec4944c770f7668ab51c4348d9c2f38
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.10.184-175.749.amzn2.x86_64
Operating System: Amazon Linux 2
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 7.612GiB
Name: ip-10-10-4-207.us-east-2.compute.internal
ID: IGJ7:R2LK:GTXA:RLRD:BBEA:7G4G:RJBX:II6O:G2HG:YYRW:23SJ:KAZT
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Buildx version
/bin/docker buildx version
github.com/docker/buildx 0.0.0+unknown
Creating a new builder instance
/bin/docker buildx create --name builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d31 --driver docker-container --use
builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d31
Booting builder
/bin/docker buildx inspect --bootstrap
#1 [internal] booting buildkit
#1 pulling image moby/buildkit:buildx-stable-1
#1 pulling image moby/buildkit:buildx-stable-1 0.2s done
#1 creating container buildx_buildkit_builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d310
#1 creating container buildx_buildkit_builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d310 0.5s done
#1 DONE 0.7s
Name: builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d31
Driver: docker-container
Last Activity: 2023-08-01 07:03:51 +0000 UTC
Nodes:
Name: builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d310
Endpoint: unix:///var/run/docker.sock
Status: running
Buildkit: v0.11.6
Platforms: linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
Inspect builder
{
"nodes": [
{
"name": "builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d310",
"endpoint": "unix:///var/run/docker.sock",
"status": "running",
"buildkit": "v0.11.6",
"platforms": "linux/amd64,linux/amd64/v2,linux/amd64/v3,linux/amd64/v4,linux/arm64,linux/riscv64,linux/ppc64le,linux/s390x,linux/386,linux/mips64le,linux/mips64,linux/arm/v7,linux/arm/v6"
}
],
"name": "builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d31",
"driver": "docker-container",
"lastActivity": "2023-08-01T07:03:51.000Z"
}
BuildKit version
builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d310: v0.11.6
.....
#14 exporting to image
#14 exporting layers
#14 exporting layers 14.9s done
#14 exporting manifest sha256:1728bb3d8f7d1b84edb2ec52388e51092ad0d36d12fef52a008b4429f5b22c34 0.0s done
#14 exporting config sha256:c3539ea4a70dde4ea0e01454c88599fc0262b8b6a55685a119f847da6128ffb7 done
#14 exporting attestation manifest sha256:102072b2361b36773c1ca495e7a0c49ad88aecff2d000224ae9378dda901c045
#14 ...
#15 [auth] aws:: luminsports/php-runtime:pull,push token for public.ecr.aws
#15 DONE 0.0s
#14 exporting to image
#14 exporting attestation manifest sha256:102072b2361b36773c1ca495e7a0c49ad88aecff2d000224ae9378dda901c045 0.0s done
#14 exporting manifest list sha256:cfd1d14f1596d8de02256ead61815688d180676940d81d3fa4feb5a5902c0fa2 done
#14 pushing layers
#14 pushing layers 8.2s done
#14 pushing manifest for public.ecr.aws/luminsports/php-runtime
#14 pushing manifest for public.ecr.aws/luminsports/php-runtime 0.6s done
#14 DONE 23.7s
.....
Run mkdir -p /tmp/digests
mkdir -p /tmp/digests
digest=""
touch "/tmp/digests/${digest#sha256:}"
shell: /bin/bash -e {0}
env:
REGISTRY_IMAGE: public.ecr.aws/luminsports/php-runtime
DOCKER_METADATA_OUTPUT_VERSION: 8.2-main
DOCKER_METADATA_OUTPUT_TAGS: public.ecr.aws/luminsports/php-runtime:8.2-main
public.ecr.aws/luminsports/php-runtime:8.2
public.ecr.aws/luminsports/php-runtime:8.2-55ee41e
DOCKER_METADATA_OUTPUT_LABELS: org.opencontainers.image.created=2023-08-01T07:03:50.110Z
org.opencontainers.image.description=
org.opencontainers.image.licenses=
org.opencontainers.image.revision=55ee41e49966302b5739eb0a054d6d6698d03316
org.opencontainers.image.source=https://github.com/luminsports/docker-php-runtime
org.opencontainers.image.title=docker-php-runtime
org.opencontainers.image.url=https://github.com/luminsports/docker-php-runtime
org.opencontainers.image.version=8.2-main
DOCKER_METADATA_OUTPUT_JSON: {"tags":["public.ecr.aws/luminsports/php-runtime:8.2-main","public.ecr.aws/luminsports/php-runtime:8.2","public.ecr.aws/luminsports/php-runtime:8.2-55ee41e"],"labels":{"org.opencontainers.image.created":"2023-08-01T07:03:50.110Z","org.opencontainers.image.description":"","org.opencontainers.image.licenses":"","org.opencontainers.image.revision":"55ee41e49966302b5739eb0a054d6d6698d03316","org.opencontainers.image.source":"https://github.com/luminsports/docker-php-runtime","org.opencontainers.image.title":"docker-php-runtime","org.opencontainers.image.url":"https://github.com/luminsports/docker-php-runtime","org.opencontainers.image.version":"8.2-main"}}
DOCKER_METADATA_OUTPUT_BAKE_FILE: /tmp/docker-actions-toolkit-KZi5wP/docker-metadata-action-bake.json
Will work on getting this repo public to make it easier to share logs.
- name: Set up Docker Buildx uses: docker/setup-buildx-action@v2 - name: Build and push Docker image id: build-docker uses: docker/build-push-action@v4 with: file: Dockerfile tags: "my-service:latest" cache-from: type=gha cache-to: type=gha,mode=max - name: Summarize the Docker digest run: | echo 'Digest: ${{ steps.build-docker.outputs.digest }}'
Looking again at your workflow this is expected. When using the docker/setup-buildx-action@v2
action without specifying the driver
, a container builder will be created. As you don't export the build result (load or push), then no digest is available in result metadata.
Depending on your use case you can either use the docker
driver in the setup buildx action or set load: true
in the build push action.
I have updated our workflow in https://github.com/docker/build-push-action/pull/928 to check this behavior. Here is a summary when digest and imageid is available following your setup:
digest | imageid | |
---|---|---|
docker (no export) |
:white_check_mark: | :white_check_mark: |
docker-container (no export) |
:x: | :x: |
docker (load) |
:white_check_mark: | :white_check_mark: |
docker-container (load) |
:warning: | :white_check_mark: |
docker (push) |
:white_check_mark: | :white_check_mark: |
docker-container (push) |
:white_check_mark: | :warning: |
:warning: means the value is set for backward compatibility.
imageid
should be used if you want to check an image from docker store (docker image ls
) anddigest
for a pushed image. More info: https://github.com/docker/build-push-action/pull/569
Hello! Original author here. 👋 I can confirm that adding load: true
made outputs.digest
be populated for me. I will leave it up to the maintainers to decide if this issue should be closed now, closed when this behaviour is documented, or when a new ticket for documenting the behaviour has been created.
Hello - it's me again! load: true
worked without push: true
, but am getting
buildx failed with: ERROR: push and load may not be set together at the moment
with push: true
. Is there a workaround for this? I saw --output ...,push=true
above, so maybe --output=type=docker,push=true
?
Hello - it's me again!
load: true
worked withoutpush: true
, but am gettingbuildx failed with: ERROR: push and load may not be set together at the moment
--load
is a shorthand for --output=type=docker
like --push
is a shorthand for --output=type=registry
but multiple exporters are currently not supported: https://github.com/moby/buildkit/issues/1555
with
push: true
. Is there a workaround for this?
You need to call the action twice as shown in https://docs.docker.com/build/ci/github-actions/test-before-push/ if this is to test your image before pushing it. Otherwise no sure why you would like to load and push.
In case it helps anyone:
My runners were on Amazon Linux 2 (also tried Amazon Linux 2023), which are stuck on Docker Server 20.x.
I updated my runners to Ubuntu 22.04, which comes with Docker Server 24.x. The digest is returned correctly now.
Be aware, that docker.io in version 24.x, which comes with Ubuntu 22.04.1+ is actually missing buildx up until at least 2023-09-07.
Hey guys! For me, it was actually a problem with the buildx version in my self-hosted GitHub Actions runner on AWS... based on Amazon Linux 2 as well.
I noticed this on the "build & push" step:
I was getting no image id, no digest and no metadata.
I went and build a new AMI, downloading a buildx release from https://github.com/docker/buildx/releases/ and installing it on /usr/local/lib/docker/cli-plugins/docker-buildx
Re-ran the Docker image build, and now I got imageid, digest & metadata!
Hello! Original author here. 👋 I can confirm that adding
load: true
madeoutputs.digest
be populated for me. I will leave it up to the maintainers to decide if this issue should be closed now, closed when this behaviour is documented, or when a new ticket for documenting the behaviour has been created.
Will look to document this case
--load
is a shorthand for--output=type=docker
like--push
is a shorthand for--output=type=registry
but multiple exporters are currently not supported: moby/buildkit#1555
Since Buildx 0.13 multi exporters can be used but we need to handle that correctly in the exporter response. Atm only the first response is returned, see https://github.com/docker/buildx/issues/2181
Posting here in case someone else has this edge case....
I ran into this problem as well and traced the problem to the Amazon Linux docker package.
When running docker info the following output is returned
Client:
Version: 25.0.3
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.0.0+unknown
Path: /usr/libexec/docker/cli-plugins/docker-buildx
...
Buildx is definitely installed with this package and I believe it is 0.13.x as the provenance param works with the cli on the server. This is a current open issue on the Amazon Linux 2023 repo.
When using this action on an Amazon Linux VM hosted runner this invalid version prevents the provenance flag from being passed to the cli and therefore no digest is generated. Manually setting provenance:true does not fix it and generates a warning in attestations stating that provenance is not allow for versions under 0.10.x
The current options are to not use this action and write the CLI command or switch to Ubuntu. I ended up terminating all our runners and recreating them on Ubuntu.
This is sonewhat edge case however it will affect attempting to use Amazon Linux as a runner when using the multi runner script referenced in the docker docs. it would be nice to have an option in the action to manually append cli arguments or bypass version checks to force through cli args as workarounds for this or future similar cases.
Troubleshooting
Before submitting a bug report please read the Troubleshooting doc. ✅
Behaviour
I have a Github actions workflow that previously has been using
build-push-action@v4
withoutsetup-buildx-action
and relying onoutputs.digest
from thebuild-push-action
(to know which Docker image to deploy). When addingthe digest is no longer output from
build-push-action@v4
and instead empty.Steps to reproduce this issue
Unfortunately what I am working on is closed source so I can't share this.
setup-buildx-action@v2
.build-push-action@v4
. In my case I am also pushing the image & have GHA caching enabled (not sure that's relevant here, though). Give the stepid: dockerbuild
.echo Digest: ${{ steps.dockerbuild.outputs.digest }}
.Expected behaviour
The echo output should contain the digest, not an empty string.
Actual behaviour
The output is
Digest:
with no digest.Configuration
Logs
Not available.