Workflow hangs with build summary

joh-klein commented 1 week ago

Contributing guidelines

[X] I've read the contributing guidelines and wholeheartedly agree

I've found a bug, and:

[X] The documentation does not mention anything about my problem
[X] There are no open or closed issues that are related to my problem

Description

I updated my workflow to use docker/build-push-action@v6 and the workflow doesn't finish anymore. It hangs at the Post-Build-Push step.

Expected behaviour

Workflow should finish

Actual behaviour

Workflow hangs at the Post "Build and push" step and times out

Repository URL

No response

Workflow run URL

No response

YAML workflow

name: Publish Docker image
on:
  push:
    branches: [ main ]
  workflow_dispatch:

jobs:
  docker-build-tag-push:
    name: Build and push Docker image to GitHub Registry
    runs-on: self-hosted
    permissions:
      contents: read
      packages: write
      attestations: write
    steps:
      - uses: actions/checkout@v4

      - name: Login to Docker Hub
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
        with:
          buildkitd-flags: --debug
          driver: kubernetes

      - name: Build and push
        uses: docker/build-push-action@v6
        with:
          context: ./runner
          push: true
          tags: ghcr.io/${{ github.repository }}:latest
          provenance: false
          sbom: false

Workflow logs

Post job cleanup.
Generating build summary
  exporting build record to /home/github/_work/_temp/docker-actions-toolkit-pm5Cga/export
  /usr/bin/mkfifo /home/github/_work/_temp/docker-actions-toolkit-pm5Cga/buildx-in-ODSj6m.fifo
  /usr/bin/mkfifo /home/github/_work/_temp/docker-actions-toolkit-pm5Cga/buildx-out-i0Xvtd.fifo
  docker buildx --builder builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a dial-stdio
  docker run --rm -i -v /home/github/.docker/buildx/refs:/buildx-refs -v /home/github/_work/_temp/docker-actions-toolkit-pm5Cga/export:/out docker.io/dockereng/export-build:latest --ref-state-dir=/buildx-refs --node=builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a/builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a0 --ref=khqxsvwt55emwi9liq51xojq7 --uid=1001 --gid=121
  Unable to find image 'dockereng/export-build:latest' locally
  latest: Pulling from dockereng/export-build
  bb431e31f450: Pulling fs layer
  03f1761d02d6: Pulling fs layer
  03f1761d02d6: Download complete
  bb431e31f450: Verifying Checksum
  bb431e31f450: Download complete
  bb431e31f450: Pull complete
  03f1761d02d6: Pull complete
  Digest: sha256:3dfedea3148487c108965dede834f22e81528fc5b2f3989e4b8ecec2f8fe10ae
  Status: Downloaded newer image for dockereng/export-build:latest
  2024/06/25 14:33:53 failed to fill local state: failed to stat local ref directory /buildx-refs/builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a/builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a0: stat /buildx-refs/builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a/builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a0/: no such file or directory
  Warning: Failed to export build record: /home/github/_work/_temp/docker-actions-toolkit-pm5Cga/export/rec.dockerbuild not found
Removing temp folder /home/github/_work/_temp/docker-actions-toolkit-9M2Ath
Post cache
  State not set

BuildKit logs

No response

Additional info

No response

crazy-max commented 1 week ago

Thanks for your feedback, can you share full logs of your workflow please?

joh-klein commented 1 week ago

2024-06-25T14:33:43.9780559Z ##[group]Run docker/build-push-action@v6
2024-06-25T14:33:43.9781040Z with:
2024-06-25T14:33:43.9781371Z   context: ./runner
2024-06-25T14:33:43.9781823Z   push: true
2024-06-25T14:33:43.9782222Z   tags: ghcr.io/[redacted]:latest
2024-06-25T14:33:43.9787846Z   provenance: false
2024-06-25T14:33:43.9788228Z   sbom: false
2024-06-25T14:33:43.9788773Z   load: false
2024-06-25T14:33:43.9789059Z   no-cache: false
2024-06-25T14:33:43.9789439Z   pull: false
2024-06-25T14:33:43.9789959Z   github-token: ***
2024-06-25T14:33:43.9793567Z ##[endgroup]
2024-06-25T14:33:44.2428345Z ##[group]GitHub Actions runtime token ACs
2024-06-25T14:33:44.2435953Z refs/heads/main: read/write
2024-06-25T14:33:44.2437477Z ##[endgroup]
2024-06-25T14:33:44.2438898Z ##[group]Docker info
2024-06-25T14:33:44.2491310Z [command]/usr/bin/docker version
2024-06-25T14:33:44.2779323Z Client: Docker Engine - Community
2024-06-25T14:33:44.2779985Z  Version:           27.0.1
2024-06-25T14:33:44.2782073Z  API version:       1.45 (downgraded from 1.46)
2024-06-25T14:33:44.2782802Z  Go version:        go1.21.11
2024-06-25T14:33:44.2783323Z  Git commit:        7fafd33
2024-06-25T14:33:44.2784264Z  Built:             Mon Jun 24 14:57:52 2024
2024-06-25T14:33:44.2784868Z  OS/Arch:           linux/amd64
2024-06-25T14:33:44.2785650Z  Context:           default
2024-06-25T14:33:44.2785967Z 
2024-06-25T14:33:44.2786587Z Server: Docker Engine - Community
2024-06-25T14:33:44.2787180Z  Engine:
2024-06-25T14:33:44.2787666Z   Version:          26.1.4
2024-06-25T14:33:44.2788285Z   API version:      1.45 (minimum version 1.24)
2024-06-25T14:33:44.2788940Z   Go version:       go1.21.11
2024-06-25T14:33:44.2789600Z   Git commit:       de5c9cf
2024-06-25T14:33:44.2790039Z   Built:            Wed Jun  5 11:28:57 2024
2024-06-25T14:33:44.2790524Z   OS/Arch:          linux/amd64
2024-06-25T14:33:44.2791032Z   Experimental:     false
2024-06-25T14:33:44.2791481Z  containerd:
2024-06-25T14:33:44.2792157Z   Version:          1.6.33
2024-06-25T14:33:44.2792691Z   GitCommit:        d2d58213f83a351ca8f528a95fbd145f5654e957
2024-06-25T14:33:44.2793281Z  runc:
2024-06-25T14:33:44.2793760Z   Version:          1.1.12
2024-06-25T14:33:44.2794604Z   GitCommit:        v1.1.12-0-g51d5e94
2024-06-25T14:33:44.2795354Z  docker-init:
2024-06-25T14:33:44.2795867Z   Version:          0.19.0
2024-06-25T14:33:44.2796325Z   GitCommit:        de40ad0
2024-06-25T14:33:44.2848667Z [command]/usr/bin/docker info
2024-06-25T14:33:44.3397315Z Client: Docker Engine - Community
2024-06-25T14:33:44.3398621Z  Version:    27.0.1
2024-06-25T14:33:44.3399022Z  Context:    default
2024-06-25T14:33:44.3399449Z  Debug Mode: false
2024-06-25T14:33:44.3400067Z  Plugins:
2024-06-25T14:33:44.3400619Z   buildx: Docker Buildx (Docker Inc.)
2024-06-25T14:33:44.3401122Z     Version:  v0.15.1
2024-06-25T14:33:44.3404768Z     Path:     /usr/libexec/docker/cli-plugins/docker-buildx
2024-06-25T14:33:44.3405946Z   compose: Docker Compose (Docker Inc.)
2024-06-25T14:33:44.3407124Z     Version:  v2.28.1
2024-06-25T14:33:44.3407703Z     Path:     /usr/libexec/docker/cli-plugins/docker-compose
2024-06-25T14:33:44.3408019Z 
2024-06-25T14:33:44.3408149Z Server:
2024-06-25T14:33:44.3408588Z  Containers: 27
2024-06-25T14:33:44.3408998Z   Running: 16
2024-06-25T14:33:44.3409362Z   Paused: 0
2024-06-25T14:33:44.3409811Z   Stopped: 11
2024-06-25T14:33:44.3410170Z  Images: 14
2024-06-25T14:33:44.3410688Z  Server Version: 26.1.4
2024-06-25T14:33:44.3411119Z  Storage Driver: overlay2
2024-06-25T14:33:44.3411535Z   Backing Filesystem: extfs
2024-06-25T14:33:44.3411972Z   Supports d_type: true
2024-06-25T14:33:44.3412399Z   Using metacopy: false
2024-06-25T14:33:44.3412801Z   Native Overlay Diff: true
2024-06-25T14:33:44.3413285Z   userxattr: false
2024-06-25T14:33:44.3413745Z  Logging Driver: json-file
2024-06-25T14:33:44.3414227Z  Cgroup Driver: systemd
2024-06-25T14:33:44.3414697Z  Cgroup Version: 2
2024-06-25T14:33:44.3415083Z  Plugins:
2024-06-25T14:33:44.3415436Z   Volume: local
2024-06-25T14:33:44.3415958Z   Network: bridge host ipvlan macvlan null overlay
2024-06-25T14:33:44.3416574Z   Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
2024-06-25T14:33:44.3417121Z  Swarm: inactive
2024-06-25T14:33:44.3417614Z  Runtimes: io.containerd.runc.v2 runc
2024-06-25T14:33:44.3418062Z  Default Runtime: runc
2024-06-25T14:33:44.3418594Z  Init Binary: docker-init
2024-06-25T14:33:44.3419027Z  containerd version: d2d58213f83a351ca8f528a95fbd145f5654e957
2024-06-25T14:33:44.3419610Z  runc version: v1.1.12-0-g51d5e94
2024-06-25T14:33:44.3420207Z  init version: de40ad0
2024-06-25T14:33:44.3420602Z  Security Options:
2024-06-25T14:33:44.3420991Z   apparmor
2024-06-25T14:33:44.3421591Z   seccomp
2024-06-25T14:33:44.3421982Z    Profile: builtin
2024-06-25T14:33:44.3422683Z   cgroupns
2024-06-25T14:33:44.3423202Z  Kernel Version: 6.8.0-35-generic
2024-06-25T14:33:44.3423587Z  Operating System: Ubuntu 24.04 LTS
2024-06-25T14:33:44.3424175Z  OSType: linux
2024-06-25T14:33:44.3424647Z  Architecture: x86_64
2024-06-25T14:33:44.3425043Z  CPUs: 8
2024-06-25T14:33:44.3425362Z  Total Memory: 15.62GiB
2024-06-25T14:33:44.3425831Z  Name: v220240684822273856
2024-06-25T14:33:44.3426320Z  ID: 09c7f958-1e44-4da9-9591-95589d729926
2024-06-25T14:33:44.3426782Z  Docker Root Dir: /var/lib/docker
2024-06-25T14:33:44.3427275Z  Debug Mode: false
2024-06-25T14:33:44.3427764Z  Experimental: false
2024-06-25T14:33:44.3428292Z  Insecure Registries:
2024-06-25T14:33:44.3428927Z   127.0.0.0/8
2024-06-25T14:33:44.3429433Z  Live Restore Enabled: false
2024-06-25T14:33:44.3429667Z 
2024-06-25T14:33:44.3430708Z ##[endgroup]
2024-06-25T14:33:44.3431371Z ##[group]Proxy configuration
2024-06-25T14:33:44.3432050Z No proxy configuration found
2024-06-25T14:33:44.3432835Z ##[endgroup]
2024-06-25T14:33:44.4207478Z ##[group]Buildx version
2024-06-25T14:33:44.4222273Z [command]/usr/bin/docker buildx version
2024-06-25T14:33:44.4853886Z github.com/docker/buildx v0.15.1 1c1dbb2
2024-06-25T14:33:44.4917627Z ##[endgroup]
2024-06-25T14:33:44.4918508Z ##[group]Builder info
2024-06-25T14:33:44.9013778Z {
2024-06-25T14:33:44.9014670Z   "nodes": [
2024-06-25T14:33:44.9015292Z     {
2024-06-25T14:33:44.9016236Z       "name": "builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a0",
2024-06-25T14:33:44.9017483Z       "endpoint": "kubernetes:///builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a?deployment=buildkit-ca4b28c0-8b00-41dd-ad2e-32d7f0fb786b-af455&kubeconfig=",
2024-06-25T14:33:44.9018588Z       "status": "running",
2024-06-25T14:33:44.9019209Z       "buildkitd-flags": "--debug --allow-insecure-entitlement=network.host",
2024-06-25T14:33:44.9019775Z       "buildkit": "v0.13.2",
2024-06-25T14:33:44.9020435Z       "platforms": "linux/amd64,linux/amd64/v2,linux/amd64/v3,linux/amd64/v4,linux/386",
2024-06-25T14:33:44.9021043Z       "features": {
2024-06-25T14:33:44.9021494Z         "Automatically load images to the Docker Engine image store": true,
2024-06-25T14:33:44.9022134Z         "Cache export": true,
2024-06-25T14:33:44.9022666Z         "Docker exporter": true,
2024-06-25T14:33:44.9023654Z         "Multi-platform build": true,
2024-06-25T14:33:44.9024220Z         "OCI exporter": true
2024-06-25T14:33:44.9024578Z       },
2024-06-25T14:33:44.9025005Z       "labels": {
2024-06-25T14:33:44.9025452Z         "org.mobyproject.buildkit.worker.executor": "oci",
2024-06-25T14:33:44.9026355Z         "org.mobyproject.buildkit.worker.hostname": "builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a0-86848fc988-sn49j",
2024-06-25T14:33:44.9027279Z         "org.mobyproject.buildkit.worker.network": "host",
2024-06-25T14:33:44.9028132Z         "org.mobyproject.buildkit.worker.oci.process-mode": "sandbox",
2024-06-25T14:33:44.9028803Z         "org.mobyproject.buildkit.worker.selinux.enabled": "false",
2024-06-25T14:33:44.9029525Z         "org.mobyproject.buildkit.worker.snapshotter": "overlayfs"
2024-06-25T14:33:44.9030076Z       },
2024-06-25T14:33:44.9030424Z       "gcPolicy": [
2024-06-25T14:33:44.9030810Z         {
2024-06-25T14:33:44.9031180Z           "all": false,
2024-06-25T14:33:44.9031581Z           "filter": [
2024-06-25T14:33:44.9032050Z             "type==source.local",
2024-06-25T14:33:44.9032562Z             "type==exec.cachemount",
2024-06-25T14:33:44.9033023Z             "type==source.git.checkout"
2024-06-25T14:33:44.9033540Z           ],
2024-06-25T14:33:44.9033911Z           "keepDuration": "48h0m0s",
2024-06-25T14:33:44.9034537Z           "keepBytes": "488.3MiB"
2024-06-25T14:33:44.9034901Z         },
2024-06-25T14:33:44.9035240Z         {
2024-06-25T14:33:44.9035688Z           "all": false,
2024-06-25T14:33:44.9036098Z           "keepDuration": "1440h0m0s",
2024-06-25T14:33:44.9036529Z           "keepBytes": "47.5GiB"
2024-06-25T14:33:44.9036935Z         },
2024-06-25T14:33:44.9037457Z         {
2024-06-25T14:33:44.9038233Z           "all": false,
2024-06-25T14:33:44.9038747Z           "keepBytes": "47.5GiB"
2024-06-25T14:33:44.9039096Z         },
2024-06-25T14:33:44.9039443Z         {
2024-06-25T14:33:44.9039876Z           "all": true,
2024-06-25T14:33:44.9040272Z           "keepBytes": "47.5GiB"
2024-06-25T14:33:44.9040682Z         }
2024-06-25T14:33:44.9041024Z       ]
2024-06-25T14:33:44.9041362Z     }
2024-06-25T14:33:44.9041705Z   ],
2024-06-25T14:33:44.9042379Z   "name": "builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a",
2024-06-25T14:33:44.9043023Z   "driver": "kubernetes",
2024-06-25T14:33:44.9043523Z   "lastActivity": "2024-06-25T14:33:40.000Z"
2024-06-25T14:33:44.9046286Z }
2024-06-25T14:33:44.9047257Z ##[endgroup]
2024-06-25T14:33:44.9869865Z [command]/usr/bin/docker buildx build --iidfile /home/github/_work/_temp/docker-actions-toolkit-9M2Ath/build-iidfile-68ed2eb6d8.txt --attest type=provenance,disabled=true --attest type=sbom,disabled=true --tag ghcr.io/[redacted]:latest --metadata-file /home/github/_work/_temp/docker-actions-toolkit-9M2Ath/build-metadata-33660b2308.json --push ./runner
2024-06-25T14:33:45.8039215Z #0 building with "builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a" instance using kubernetes driver
2024-06-25T14:33:45.8039829Z 
2024-06-25T14:33:45.8040093Z #1 [internal] load build definition from Dockerfile
2024-06-25T14:33:45.8040654Z #1 transferring dockerfile: 3.58kB 0.1s done
2024-06-25T14:33:45.8041169Z #1 DONE 0.1s
2024-06-25T14:33:45.8041415Z 
2024-06-25T14:33:45.8041644Z #2 [internal] load metadata for docker.io/library/ubuntu:22.04
2024-06-25T14:33:47.2049925Z #2 DONE 1.6s
2024-06-25T14:33:47.3176893Z 
2024-06-25T14:33:47.3177700Z #3 [internal] load .dockerignore
2024-06-25T14:33:47.3178428Z #3 transferring context: 2B 0.1s done
2024-06-25T14:33:47.3179272Z #3 DONE 0.1s
2024-06-25T14:33:47.3179566Z 
2024-06-25T14:33:47.3179857Z #4 importing cache manifest from s3:17919850164124835009
2024-06-25T14:33:47.4491841Z #4 DONE 0.1s
2024-06-25T14:33:47.4492439Z 
2024-06-25T14:33:47.4493085Z #5 [ 1/14] FROM docker.io/library/ubuntu:22.04@sha256:19478ce7fc2ffbce89df29fea5725a8d12e57de52eb9ea570890dc5852aac1ac
2024-06-25T14:33:47.4494428Z #5 resolve docker.io/library/ubuntu:22.04@sha256:19478ce7fc2ffbce89df29fea5725a8d12e57de52eb9ea570890dc5852aac1ac done
2024-06-25T14:33:47.4495453Z #5 DONE 0.0s
2024-06-25T14:33:47.4495855Z 
2024-06-25T14:33:47.4496011Z #6 [internal] load build context
2024-06-25T14:33:47.4496505Z #6 transferring context: 1.18kB 0.1s done
2024-06-25T14:33:47.5902096Z #6 DONE 0.1s
2024-06-25T14:33:47.5902485Z 
2024-06-25T14:33:47.5904407Z #7 [ 8/14] RUN [redacted]
2024-06-25T14:33:47.5905958Z #7 CACHED
2024-06-25T14:33:47.5906199Z 
2024-06-25T14:33:47.5907639Z #8 [ 3/14] RUN [redacted]
2024-06-25T14:33:47.5909398Z #8 CACHED
2024-06-25T14:33:47.5909668Z 
2024-06-25T14:33:47.5910901Z #9 [ 9/14] RUN [redacted]
2024-06-25T14:33:47.5912131Z #9 CACHED
2024-06-25T14:33:47.5912288Z 
2024-06-25T14:33:47.5912686Z #10 [12/14] RUN apt-get clean && rm -rf /var/lib/apt/lists/
2024-06-25T14:33:47.5913150Z #10 CACHED
2024-06-25T14:33:47.5913307Z 
2024-06-25T14:33:47.5914462Z #11 [ 5/14] RUN [redacted]
2024-06-25T14:33:47.5916009Z #11 CACHED
2024-06-25T14:33:47.5916311Z 
2024-06-25T14:33:47.5916500Z #12 [ 7/14] WORKDIR /home/github
2024-06-25T14:33:47.5917027Z #12 CACHED
2024-06-25T14:33:47.5917181Z 
2024-06-25T14:33:47.5918678Z #13 [11/14] RUN [redacted]
2024-06-25T14:33:47.5920013Z #13 CACHED
2024-06-25T14:33:47.5920169Z 
2024-06-25T14:33:47.5920382Z #14 [13/14] COPY entrypoint.sh .
2024-06-25T14:33:47.5920795Z #14 CACHED
2024-06-25T14:33:47.5921003Z 
2024-06-25T14:33:47.5921504Z #15 [ 4/14] RUN [redacted]
2024-06-25T14:33:47.5922188Z #15 CACHED
2024-06-25T14:33:47.5922479Z 
2024-06-25T14:33:47.5923681Z #16 [ 6/14] RUN [redacted]
2024-06-25T14:33:47.5925103Z #16 CACHED
2024-06-25T14:33:47.5925257Z 
2024-06-25T14:33:47.5926173Z #17 [ 2/14] RUN [redacted]
2024-06-25T14:33:47.5927063Z #17 CACHED
2024-06-25T14:33:47.5927215Z 
2024-06-25T14:33:47.5927894Z #18 [10/14] RUN [redacted]
2024-06-25T14:33:47.5928808Z #18 CACHED
2024-06-25T14:33:47.5929018Z 
2024-06-25T14:33:47.5929183Z #19 [14/14] RUN chmod +x entrypoint.sh
2024-06-25T14:33:47.5929593Z #19 CACHED
2024-06-25T14:33:47.7636353Z 
2024-06-25T14:33:47.7637044Z #20 exporting to image
2024-06-25T14:33:47.7637690Z #20 exporting layers done
2024-06-25T14:33:47.7638355Z #20 exporting manifest sha256:fd42cd18f5a19222cef43847f6cee1a610b3ba0cfd56e5323963b4626d164a6f 0.0s done
2024-06-25T14:33:47.7639241Z #20 exporting config sha256:3594e57155df0a70af8384445c4412678536273c89ee758503797e097026c1bc done
2024-06-25T14:33:47.7639949Z #20 pushing layers
2024-06-25T14:33:47.9639218Z #20 ...
2024-06-25T14:33:47.9639564Z 
2024-06-25T14:33:47.9640383Z #21 [auth] [redacted]:pull,push token for ghcr.io
2024-06-25T14:33:47.9641302Z #21 DONE 0.0s
2024-06-25T14:33:48.1136368Z 
2024-06-25T14:33:48.1137063Z #20 exporting to image
2024-06-25T14:33:49.2321076Z #20 pushing layers 1.6s done
2024-06-25T14:33:49.2322654Z #20 pushing manifest for ghcr.io/[redacted]:latest@sha256:fd42cd18f5a19222cef43847f6cee1a610b3ba0cfd56e5323963b4626d164a6f
2024-06-25T14:33:49.8968656Z #20 pushing manifest for ghcr.io/[redacted]:latest@sha256:fd42cd18f5a19222cef43847f6cee1a610b3ba0cfd56e5323963b4626d164a6f 0.7s done
2024-06-25T14:33:49.8970043Z #20 DONE 2.3s
2024-06-25T14:33:49.8970484Z 
2024-06-25T14:33:49.8970687Z #22 exporting cache to Amazon S3
2024-06-25T14:33:49.8971144Z #22 preparing build cache for export
2024-06-25T14:33:50.2113787Z #22 preparing build cache for export 0.3s done
2024-06-25T14:33:50.2114569Z #22 DONE 0.3s
2024-06-25T14:33:50.3973725Z 
2024-06-25T14:33:50.3975294Z #23 resolving provenance for metadata file
2024-06-25T14:33:50.3976072Z #23 DONE 0.1s
2024-06-25T14:33:50.4289780Z ##[group]ImageID
2024-06-25T14:33:50.4290568Z sha256:3594e57155df0a70af8384445c4412678536273c89ee758503797e097026c1bc
2024-06-25T14:33:50.4291680Z ##[endgroup]
2024-06-25T14:33:50.4292411Z ##[group]Digest
2024-06-25T14:33:50.4292886Z sha256:fd42cd18f5a19222cef43847f6cee1a610b3ba0cfd56e5323963b4626d164a6f
2024-06-25T14:33:50.4293778Z ##[endgroup]
2024-06-25T14:33:50.4294706Z ##[group]Metadata
2024-06-25T14:33:50.4295308Z {
2024-06-25T14:33:50.4295639Z   "buildx.build.provenance": {
2024-06-25T14:33:50.4296181Z     "buildType": "https://mobyproject.org/buildkit@v1",
2024-06-25T14:33:50.4296748Z     "materials": [
2024-06-25T14:33:50.4297165Z       {
2024-06-25T14:33:50.4297672Z         "uri": "pkg:docker/ubuntu@22.04?platform=linux%2Famd64",
2024-06-25T14:33:50.4298110Z         "digest": {
2024-06-25T14:33:50.4298783Z           "sha256": "19478ce7fc2ffbce89df29fea5725a8d12e57de52eb9ea570890dc5852aac1ac"
2024-06-25T14:33:50.4299560Z         }
2024-06-25T14:33:50.4299926Z       }
2024-06-25T14:33:50.4300264Z     ],
2024-06-25T14:33:50.4300642Z     "invocation": {
2024-06-25T14:33:50.4301042Z       "configSource": {
2024-06-25T14:33:50.4301447Z         "entryPoint": "Dockerfile"
2024-06-25T14:33:50.4301948Z       },
2024-06-25T14:33:50.4302303Z       "parameters": {
2024-06-25T14:33:50.4302638Z         "frontend": "dockerfile.v0",
2024-06-25T14:33:50.4308284Z         "locals": [
2024-06-25T14:33:50.4308585Z           {
2024-06-25T14:33:50.4308952Z             "name": "context"
2024-06-25T14:33:50.4309433Z           },
2024-06-25T14:33:50.4309797Z           {
2024-06-25T14:33:50.4310116Z             "name": "dockerfile"
2024-06-25T14:33:50.4310603Z           }
2024-06-25T14:33:50.4310947Z         ]
2024-06-25T14:33:50.4311306Z       },
2024-06-25T14:33:50.4311725Z       "environment": {
2024-06-25T14:33:50.4312069Z         "platform": "linux/amd64"
2024-06-25T14:33:50.4312530Z       }
2024-06-25T14:33:50.4312937Z     }
2024-06-25T14:33:50.4313293Z   },
2024-06-25T14:33:50.4314227Z   "buildx.build.ref": "builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a/builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a0/khqxsvwt55emwi9liq51xojq7",
2024-06-25T14:33:50.4315391Z   "containerimage.config.digest": "sha256:3594e57155df0a70af8384445c4412678536273c89ee758503797e097026c1bc",
2024-06-25T14:33:50.4316084Z   "containerimage.descriptor": {
2024-06-25T14:33:50.4316637Z     "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
2024-06-25T14:33:50.4317428Z     "digest": "sha256:fd42cd18f5a19222cef43847f6cee1a610b3ba0cfd56e5323963b4626d164a6f",
2024-06-25T14:33:50.4318034Z     "size": 3065,
2024-06-25T14:33:50.4318344Z     "platform": {
2024-06-25T14:33:50.4318812Z       "architecture": "amd64",
2024-06-25T14:33:50.4319213Z       "os": "linux"
2024-06-25T14:33:50.4319645Z     }
2024-06-25T14:33:50.4320006Z   },
2024-06-25T14:33:50.4320514Z   "containerimage.digest": "sha256:fd42cd18f5a19222cef43847f6cee1a610b3ba0cfd56e5323963b4626d164a6f",
2024-06-25T14:33:50.4321746Z   "image.name": "ghcr.io/[redacted]:latest"
2024-06-25T14:33:50.4322212Z }
2024-06-25T14:33:50.4323319Z ##[endgroup]
2024-06-25T14:33:50.4324022Z ##[group]Reference
2024-06-25T14:33:50.4325602Z builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a/builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a0/khqxsvwt55emwi9liq51xojq7
2024-06-25T14:33:50.4326645Z ##[endgroup]

joh-klein commented 1 week ago

There are some ARGs and ENVs I left out (and also just tried without)

crazy-max commented 6 days ago

2024-06-25T14:33:44.2779323Z Client: Docker Engine - Community
2024-06-25T14:33:44.2779985Z  Version:           27.0.1
...
2024-06-25T14:33:44.2786587Z Server: Docker Engine - Community
2024-06-25T14:33:44.2787180Z  Engine:
2024-06-25T14:33:44.2787666Z   Version:          26.1.4

Seems Docker client/server version is not aligned on your self-hosted environment but don't think that's related.

There are some ARGs and ENVs I left out (and also just tried without)

Thanks that's fine, can you share logs of "Post Set up Docker Buildx" step as well?

joh-klein commented 6 days ago

Logs for the "Post Set up Docker Buildx" step

2024-06-25T14:33:40.3619043Z ##[group]Run docker/setup-buildx-action@v3
2024-06-25T14:33:40.3619380Z with:
2024-06-25T14:33:40.3619592Z   buildkitd-flags: --debug
2024-06-25T14:33:40.3619834Z   driver: kubernetes
2024-06-25T14:33:40.3620049Z   install: false
2024-06-25T14:33:40.3620256Z   use: true
2024-06-25T14:33:40.3620454Z   cache-binary: true
2024-06-25T14:33:40.3620673Z   cleanup: true
2024-06-25T14:33:40.3620879Z ##[endgroup]
2024-06-25T14:33:40.5104332Z ##[group]Docker info
2024-06-25T14:33:40.5118261Z [command]/usr/bin/docker version
2024-06-25T14:33:40.5396611Z Client: Docker Engine - Community
2024-06-25T14:33:40.5397433Z  Version:           27.0.1
2024-06-25T14:33:40.5397733Z  API version:       1.45 (downgraded from 1.46)
2024-06-25T14:33:40.5398114Z  Go version:        go1.21.11
2024-06-25T14:33:40.5398390Z  Git commit:        7fafd33
2024-06-25T14:33:40.5398681Z  Built:             Mon Jun 24 14:57:52 2024
2024-06-25T14:33:40.5399043Z  OS/Arch:           linux/amd64
2024-06-25T14:33:40.5399298Z  Context:           default
2024-06-25T14:33:40.5399535Z 
2024-06-25T14:33:40.5399687Z Server: Docker Engine - Community
2024-06-25T14:33:40.5406654Z  Engine:
2024-06-25T14:33:40.5406914Z   Version:          26.1.4
2024-06-25T14:33:40.5407241Z   API version:      1.45 (minimum version 1.24)
2024-06-25T14:33:40.5407556Z   Go version:       go1.21.11
2024-06-25T14:33:40.5407811Z   Git commit:       de5c9cf
2024-06-25T14:33:40.5408084Z   Built:            Wed Jun  5 11:28:57 2024
2024-06-25T14:33:40.5408392Z   OS/Arch:          linux/amd64
2024-06-25T14:33:40.5408653Z   Experimental:     false
2024-06-25T14:33:40.5408898Z  containerd:
2024-06-25T14:33:40.5409102Z   Version:          1.6.33
2024-06-25T14:33:40.5409421Z   GitCommit:        d2d58213f83a351ca8f528a95fbd145f5654e957
2024-06-25T14:33:40.5409751Z  runc:
2024-06-25T14:33:40.5409943Z   Version:          1.1.12
2024-06-25T14:33:40.5410297Z   GitCommit:        v1.1.12-0-g51d5e94
2024-06-25T14:33:40.5410936Z  docker-init:
2024-06-25T14:33:40.5411154Z   Version:          0.19.0
2024-06-25T14:33:40.5411401Z   GitCommit:        de40ad0
2024-06-25T14:33:40.5425895Z [command]/usr/bin/docker info
2024-06-25T14:33:40.5949501Z Client: Docker Engine - Community
2024-06-25T14:33:40.5950769Z  Version:    27.0.1
2024-06-25T14:33:40.5953692Z  Context:    default
2024-06-25T14:33:40.5954015Z  Debug Mode: false
2024-06-25T14:33:40.5954371Z  Plugins:
2024-06-25T14:33:40.5954607Z   buildx: Docker Buildx (Docker Inc.)
2024-06-25T14:33:40.5954904Z     Version:  v0.15.1
2024-06-25T14:33:40.5955344Z     Path:     /usr/libexec/docker/cli-plugins/docker-buildx
2024-06-25T14:33:40.5955704Z   compose: Docker Compose (Docker Inc.)
2024-06-25T14:33:40.5956002Z     Version:  v2.28.1
2024-06-25T14:33:40.5956343Z     Path:     /usr/libexec/docker/cli-plugins/docker-compose
2024-06-25T14:33:40.5956596Z 
2024-06-25T14:33:40.5956681Z Server:
2024-06-25T14:33:40.5956863Z  Containers: 25
2024-06-25T14:33:40.5957103Z   Running: 14
2024-06-25T14:33:40.5957305Z   Paused: 0
2024-06-25T14:33:40.5957501Z   Stopped: 11
2024-06-25T14:33:40.5957714Z  Images: 14
2024-06-25T14:33:40.5957919Z  Server Version: 26.1.4
2024-06-25T14:33:40.5958161Z  Storage Driver: overlay2
2024-06-25T14:33:40.5958397Z   Backing Filesystem: extfs
2024-06-25T14:33:40.5958646Z   Supports d_type: true
2024-06-25T14:33:40.5958875Z   Using metacopy: false
2024-06-25T14:33:40.5959100Z   Native Overlay Diff: true
2024-06-25T14:33:40.5959333Z   userxattr: false
2024-06-25T14:33:40.5959574Z  Logging Driver: json-file
2024-06-25T14:33:40.5959818Z  Cgroup Driver: systemd
2024-06-25T14:33:40.5960050Z  Cgroup Version: 2
2024-06-25T14:33:40.5960257Z  Plugins:
2024-06-25T14:33:40.5960444Z   Volume: local
2024-06-25T14:33:40.5960704Z   Network: bridge host ipvlan macvlan null overlay
2024-06-25T14:33:40.5961190Z   Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
2024-06-25T14:33:40.5961579Z  Swarm: inactive
2024-06-25T14:33:40.5961820Z  Runtimes: runc io.containerd.runc.v2
2024-06-25T14:33:40.5962098Z  Default Runtime: runc
2024-06-25T14:33:40.5963040Z  Init Binary: docker-init
2024-06-25T14:33:40.5963369Z  containerd version: d2d58213f83a351ca8f528a95fbd145f5654e957
2024-06-25T14:33:40.5963855Z  runc version: v1.1.12-0-g51d5e94
2024-06-25T14:33:40.5964233Z  init version: de40ad0
2024-06-25T14:33:40.5964451Z  Security Options:
2024-06-25T14:33:40.5964664Z   apparmor
2024-06-25T14:33:40.5964843Z   seccomp
2024-06-25T14:33:40.5965040Z    Profile: builtin
2024-06-25T14:33:40.5965249Z   cgroupns
2024-06-25T14:33:40.5965494Z  Kernel Version: 6.8.0-35-generic
2024-06-25T14:33:40.5965771Z  Operating System: Ubuntu 24.04 LTS
2024-06-25T14:33:40.5966037Z  OSType: linux
2024-06-25T14:33:40.5966259Z  Architecture: x86_64
2024-06-25T14:33:40.5966476Z  CPUs: 8
2024-06-25T14:33:40.5966675Z  Total Memory: 15.62GiB
2024-06-25T14:33:40.5966921Z  Name: v220240684822273856
2024-06-25T14:33:40.5967208Z  ID: 09c7f958-1e44-4da9-9591-95589d729926
2024-06-25T14:33:40.5967507Z  Docker Root Dir: /var/lib/docker
2024-06-25T14:33:40.5967762Z  Debug Mode: false
2024-06-25T14:33:40.5967981Z  Experimental: false
2024-06-25T14:33:40.5968199Z  Insecure Registries:
2024-06-25T14:33:40.5968458Z   127.0.0.0/8
2024-06-25T14:33:40.5968751Z  Live Restore Enabled: false
2024-06-25T14:33:40.5968953Z 
2024-06-25T14:33:40.5985017Z ##[endgroup]
2024-06-25T14:33:40.6692768Z ##[group]Buildx version
2024-06-25T14:33:40.6706688Z [command]/usr/bin/docker buildx version
2024-06-25T14:33:40.7275982Z github.com/docker/buildx v0.15.1 1c1dbb2
2024-06-25T14:33:40.7328856Z ##[endgroup]
2024-06-25T14:33:40.7338033Z ##[group]Creating a new builder instance
2024-06-25T14:33:40.8032498Z [command]/usr/bin/docker buildx create --name builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a --driver kubernetes --buildkitd-flags --debug --use
2024-06-25T14:33:40.9120031Z builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a
2024-06-25T14:33:40.9177245Z ##[endgroup]
2024-06-25T14:33:40.9177713Z ##[group]Booting builder
2024-06-25T14:33:40.9193410Z [command]/usr/bin/docker buildx inspect --bootstrap --builder builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a
2024-06-25T14:33:41.0272034Z #1 [internal] booting buildkit
2024-06-25T14:33:41.2117066Z #1 waiting for 1 pods to be ready, timeout: 2 minutes
2024-06-25T14:33:43.1950073Z #1 waiting for 1 pods to be ready, timeout: 2 minutes 2.1s done
2024-06-25T14:33:43.1950673Z #1 DONE 2.2s
2024-06-25T14:33:43.5112971Z Name:          builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a
2024-06-25T14:33:43.5113475Z Driver:        kubernetes
2024-06-25T14:33:43.5113797Z Last Activity: 2024-06-25 14:33:40 +0000 UTC
2024-06-25T14:33:43.5114179Z 
2024-06-25T14:33:43.5114260Z Nodes:
2024-06-25T14:33:43.5114674Z Name:                  builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a0
2024-06-25T14:33:43.5115712Z Endpoint:              kubernetes:///builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a?deployment=buildkit-ca4b28c0-8b00-41dd-ad2e-32d7f0fb786b-af455&kubeconfig=
2024-06-25T14:33:43.5116415Z Status:                running
2024-06-25T14:33:43.5116917Z BuildKit daemon flags: --debug --allow-insecure-entitlement=network.host
2024-06-25T14:33:43.5117374Z BuildKit version:      v0.13.2
2024-06-25T14:33:43.5117927Z Platforms:             linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/386
2024-06-25T14:33:43.5118354Z Labels:
2024-06-25T14:33:43.5118645Z  org.mobyproject.buildkit.worker.executor:         oci
2024-06-25T14:33:43.5119673Z  org.mobyproject.buildkit.worker.hostname:         builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a0-86848fc988-sn49j
2024-06-25T14:33:43.5120463Z  org.mobyproject.buildkit.worker.network:          host
2024-06-25T14:33:43.5121014Z  org.mobyproject.buildkit.worker.oci.process-mode: sandbox
2024-06-25T14:33:43.5121495Z  org.mobyproject.buildkit.worker.selinux.enabled:  false
2024-06-25T14:33:43.5121965Z  org.mobyproject.buildkit.worker.snapshotter:      overlayfs
2024-06-25T14:33:43.5122351Z GC Policy rule#0:
2024-06-25T14:33:43.5122586Z  All:           false
2024-06-25T14:33:43.5123361Z  Filters:       type==source.local,type==exec.cachemount,type==source.git.checkout
2024-06-25T14:33:43.5124703Z  Keep Duration: 48h0m0s
2024-06-25T14:33:43.5124955Z  Keep Bytes:    488.3MiB
2024-06-25T14:33:43.5125195Z GC Policy rule#1:
2024-06-25T14:33:43.5125424Z  All:           false
2024-06-25T14:33:43.5125658Z  Keep Duration: 1440h0m0s
2024-06-25T14:33:43.5125906Z  Keep Bytes:    47.5GiB
2024-06-25T14:33:43.5126438Z GC Policy rule#2:
2024-06-25T14:33:43.5126651Z  All:        false
2024-06-25T14:33:43.5127002Z  Keep Bytes: 47.5GiB
2024-06-25T14:33:43.5127229Z GC Policy rule#3:
2024-06-25T14:33:43.5127443Z  All:        true
2024-06-25T14:33:43.5127635Z  Keep Bytes: 47.5GiB
2024-06-25T14:33:43.5194599Z ##[endgroup]
2024-06-25T14:33:43.9551561Z ##[group]Inspect builder
2024-06-25T14:33:43.9555988Z {
2024-06-25T14:33:43.9556507Z   "nodes": [
2024-06-25T14:33:43.9556795Z     {
2024-06-25T14:33:43.9557432Z       "name": "builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a0",
2024-06-25T14:33:43.9558366Z       "endpoint": "kubernetes:///builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a?deployment=buildkit-ca4b28c0-8b00-41dd-ad2e-32d7f0fb786b-af455&kubeconfig=",
2024-06-25T14:33:43.9559127Z       "status": "running",
2024-06-25T14:33:43.9559691Z       "buildkitd-flags": "--debug --allow-insecure-entitlement=network.host",
2024-06-25T14:33:43.9560098Z       "buildkit": "v0.13.2",
2024-06-25T14:33:43.9560512Z       "platforms": "linux/amd64,linux/amd64/v2,linux/amd64/v3,linux/amd64/v4,linux/386",
2024-06-25T14:33:43.9560923Z       "features": {
2024-06-25T14:33:43.9561266Z         "Automatically load images to the Docker Engine image store": true,
2024-06-25T14:33:43.9561665Z         "Cache export": true,
2024-06-25T14:33:43.9561925Z         "Docker exporter": true,
2024-06-25T14:33:43.9562233Z         "Multi-platform build": true,
2024-06-25T14:33:43.9562522Z         "OCI exporter": true
2024-06-25T14:33:43.9562757Z       },
2024-06-25T14:33:43.9563244Z       "labels": {
2024-06-25T14:33:43.9563553Z         "org.mobyproject.buildkit.worker.executor": "oci",
2024-06-25T14:33:43.9565275Z         "org.mobyproject.buildkit.worker.hostname": "builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a0-86848fc988-sn49j",
2024-06-25T14:33:43.9565936Z         "org.mobyproject.buildkit.worker.network": "host",
2024-06-25T14:33:43.9566461Z         "org.mobyproject.buildkit.worker.oci.process-mode": "sandbox",
2024-06-25T14:33:43.9567185Z         "org.mobyproject.buildkit.worker.selinux.enabled": "false",
2024-06-25T14:33:43.9567780Z         "org.mobyproject.buildkit.worker.snapshotter": "overlayfs"
2024-06-25T14:33:43.9568150Z       },
2024-06-25T14:33:43.9568358Z       "gcPolicy": [
2024-06-25T14:33:43.9568566Z         {
2024-06-25T14:33:43.9568759Z           "all": false,
2024-06-25T14:33:43.9568984Z           "filter": [
2024-06-25T14:33:43.9569215Z             "type==source.local",
2024-06-25T14:33:43.9569489Z             "type==exec.cachemount",
2024-06-25T14:33:43.9569766Z             "type==source.git.checkout"
2024-06-25T14:33:43.9570035Z           ],
2024-06-25T14:33:43.9570256Z           "keepDuration": "48h0m0s",
2024-06-25T14:33:43.9570535Z           "keepBytes": "488.3MiB"
2024-06-25T14:33:43.9570783Z         },
2024-06-25T14:33:43.9570982Z         {
2024-06-25T14:33:43.9571193Z           "all": false,
2024-06-25T14:33:43.9571429Z           "keepDuration": "1440h0m0s",
2024-06-25T14:33:43.9571708Z           "keepBytes": "47.5GiB"
2024-06-25T14:33:43.9571963Z         },
2024-06-25T14:33:43.9572137Z         {
2024-06-25T14:33:43.9572321Z           "all": false,
2024-06-25T14:33:43.9572557Z           "keepBytes": "47.5GiB"
2024-06-25T14:33:43.9572804Z         },
2024-06-25T14:33:43.9572974Z         {
2024-06-25T14:33:43.9573172Z           "all": true,
2024-06-25T14:33:43.9573406Z           "keepBytes": "47.5GiB"
2024-06-25T14:33:43.9573639Z         }
2024-06-25T14:33:43.9573813Z       ]
2024-06-25T14:33:43.9574001Z     }
2024-06-25T14:33:43.9574649Z   ],
2024-06-25T14:33:43.9575067Z   "name": "builder-0cab9f90-307e-4cc5-af4f-e7ed6ff3356a",
2024-06-25T14:33:43.9575446Z   "driver": "kubernetes",
2024-06-25T14:33:43.9575796Z   "lastActivity": "2024-06-25T14:33:40.000Z"
2024-06-25T14:33:43.9576502Z }
2024-06-25T14:33:43.9577006Z ##[endgroup]

crazy-max commented 6 days ago

Logs for the "Post Set up Docker Buildx" step

This does not look like the logs for "Post Set up Docker Buildx" step but "Set up Docker Buildx". It should look like this: https://github.com/docker/setup-buildx-action/actions/runs/9677658877/job/26699794985#step:11:3

joh-klein commented 6 days ago

True - the job hangs at the "Post Build and push" step and I cancel it (after ~20min), so there is no "Post …" anything

crazy-max commented 6 days ago

True - the job hangs at the "Post Build and push" step and I cancel it (after ~20min), so there is no "Post …" anything

Ah damn ok, I will try to repro on my side with Docker 27.0.1 and let you know. This might be the issue.

joh-klein commented 6 days ago

I'm pretty sure it already happend with 26.1.4 … sorry :(

kkopachev commented 6 days ago

happening to me too with github hosted runners. Stuck on "Post Build Image" step. I had to cancel it, here is raw log snippet from the very end:

2024-06-26T19:57:56.7540395Z ##[endgroup]
2024-06-26T19:57:56.7541058Z ##[group]Reference
2024-06-26T19:57:56.7541894Z builder-88a33921-1310-4c79-87eb-d05c690e1200/builder-88a33921-1310-4c79-87eb-d05c690e12000/qnj71xa7baaf85s8zmn0u5k2u
2024-06-26T19:57:56.7542948Z ##[endgroup]
2024-06-26T19:57:56.7644149Z Post job cleanup.
2024-06-26T19:57:56.9844316Z ##[group]Generating build summary
2024-06-26T19:57:57.0510611Z exporting build record to /home/runner/work/_temp/docker-actions-toolkit-ghNl2O/export
2024-06-26T19:57:57.0538486Z [command]/usr/bin/mkfifo /home/runner/work/_temp/docker-actions-toolkit-ghNl2O/buildx-in-2b0iNa.fifo
2024-06-26T19:57:57.0595412Z [command]/usr/bin/mkfifo /home/runner/work/_temp/docker-actions-toolkit-ghNl2O/buildx-out-2aIDNI.fifo
2024-06-26T19:57:57.0696207Z [command]docker buildx --builder builder-88a33921-1310-4c79-87eb-d05c690e1200 dial-stdio
2024-06-26T19:57:57.0710287Z [command]docker run --rm -i -v /home/runner/.docker/buildx/refs:/buildx-refs -v /home/runner/work/_temp/docker-actions-toolkit-ghNl2O/export:/out docker.io/dockereng/export-build:latest --ref-state-dir=/buildx-refs --node=builder-88a33921-1310-4c79-87eb-d05c690e1200/builder-88a33921-1310-4c79-87eb-d05c690e12000 --ref=qnj71xa7baaf85s8zmn0u5k2u --uid=1001 --gid=127
2024-06-26T19:57:57.0993548Z Unable to find image 'dockereng/export-build:latest' locally
2024-06-26T19:57:57.3431283Z latest: Pulling from dockereng/export-build
2024-06-26T19:57:57.4114872Z bb431e31f450: Pulling fs layer
2024-06-26T19:57:57.4115568Z 03f1761d02d6: Pulling fs layer
2024-06-26T19:57:57.5320206Z 03f1761d02d6: Download complete
2024-06-26T19:57:57.9063610Z bb431e31f450: Download complete
2024-06-26T19:57:58.0688805Z bb431e31f450: Pull complete
2024-06-26T19:57:58.1097254Z 03f1761d02d6: Pull complete
2024-06-26T19:57:58.1140852Z Digest: sha256:3dfedea3148487c108965dede834f22e81528fc5b2f3989e4b8ecec2f8fe10ae
2024-06-26T19:57:58.1154591Z Status: Downloaded newer image for dockereng/export-build:latest
2024-06-26T19:57:58.7106377Z Parsing /home/runner/work/_temp/docker-actions-toolkit-ghNl2O/export/summary.json
2024-06-26T19:57:58.7108985Z Build record exported to /home/runner/work/_temp/docker-actions-toolkit-ghNl2O/export/ORG~REPO~QNJ71X.dockerbuild (26.88 KB)
2024-06-26T19:57:58.7137410Z Uploading ORG~REPO~QNJ71X.dockerbuild to blob storage
2024-06-26T19:57:58.8072271Z Beginning upload of artifact content to blob storage
2024-06-26T19:57:58.8159220Z Uploaded bytes 27526
2024-06-26T19:57:58.8437281Z Finished uploading artifact content to blob storage!
2024-06-26T19:57:58.8443956Z SHA256 hash of uploaded artifact is db9744e893bc34e948543cc59cc3e7a6eb233f99bccd3c1940ecb9782edd12fb
2024-06-26T19:57:58.8445540Z Finalizing artifact upload
2024-06-26T19:57:58.9430202Z Artifact successfully finalized (1641974970)
2024-06-26T19:57:58.9432282Z Artifact download URL: https://github.com/ORG/REPO

UI was also showing something like "State not found" at the very last line. Was stuck for 30 minutes and took a while to cancel the job.

crazy-max commented 5 days ago

@kkopachev Thanks for your feedback

happening to me too with github hosted runners

Public runners I guess then? Or self-hosted?

Stuck on "Post Build Image" step. I had to cancel it, here is raw log snippet from the very end:

So it hangs at Artifact download URL: https://github.com/ORG/REPO right? Which means issue would be around writing the summary in https://github.com/docker/build-push-action/blob/15560696de535e4014efeff63c48f16952e52dd1/src/main.ts#L165-L169

Could you share full logs of your workflow please and the workflow YAML as well? That would help.

In the meantime your can disable summary generation with DOCKER_BUILD_NO_SUMMARY env var:

      -
        name: Build and push
        uses: docker/build-push-action@v6
        with:
          push: true
          tags: user/app:latest
        env:
          DOCKER_BUILD_NO_SUMMARY: true

Or globally within your workflow if you have multiple calls to the action:

name: ci

on:
  push:
    branches:
      - 'main'

env:
  DOCKER_BUILD_NO_SUMMARY: true

kkopachev commented 5 days ago

We're using public GHA runners (x86-64), but building ARM images using kubernetes driver.

Even cancellation takes long time

Workflow files are like this:

workflow.yaml

```yaml name: Review Deploy on: pull_request: jobs: build_image: name: Build Image uses: %ORG%/%REPO%/.github/workflows/build-image.yaml@v1.20.2 with: aws-role: ${{ vars.AWS_ROLE }} aws-role-session-name: session-name kubernetes-cluster: something-staging registry: 1234567890.dkr.ecr.us-east-1.amazonaws.com/app secrets: build-secrets: | ${{ format('gh_token={0}', secrets.ACCESS_TOKEN) }} ```

and then reusable workflow referenced is quite generic, so I snipped just relevant parts and omitted eks connection, aws auth and such

reusable-workflow.yaml

```yaml jobs: build: name: Build and Push Image runs-on: ubuntu-latest outputs: ref_name: ${{ steps.ref_name.outputs.ref_name }} sha_short: ${{ steps.get_commit.outputs.sha_short }} steps: - name: Checkout uses: actions/checkout@v4 with: ref: ${{ inputs.ref }} # snip-snip - name: Prepare Buildx Driver Opts uses: actions/github-script@v7 id: driver-opts with: script: | const platforms = `${{ inputs.platforms }}`; let driverOpts = ""; if (platforms.includes('linux/amd64')) { const random = Math.floor((Math.random() * 1000000) + 1); driverOpts = ` - driver-opts: - namespace=${{ inputs.kubernetes-namespace }} - "nodeselector=kubernetes.io/arch=amd64" platforms: linux/amd64 name: arm64-${{ steps.get_commit.outputs.sha_short }}-${random}`; } core.setOutput('driver-opts', driverOpts); - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 with: driver: kubernetes driver-opts: | namespace=${{ inputs.kubernetes-namespace }} "nodeselector=kubernetes.io/arch=arm64" platforms: linux/arm64 append: ${{ steps.driver-opts.outputs.driver-opts }} - name: Build Image uses: docker/build-push-action@v6 with: build-args: ${{ inputs.build-args }} cache-from: | type=registry,ref=${{ inputs.registry }}:latest ${{ steps.ref_name.outputs.ref_name && format('type=registry,ref={0}:{1}', inputs.registry, steps.ref_name.outputs.ref_name) || '' }} cache-to: type=inline context: ${{ inputs.context }} file: ${{ inputs.file }} labels: | org.opencontainers.image.revision=${{ github.sha }} org.opencontainers.image.source=github.com/${{ github.repository }} ${{ inputs.labels }} platforms: ${{ inputs.platforms }} push: true provenance: false secrets: ${{ secrets.build-secrets }} tags: | ${{ inputs.registry }}:${{ steps.get_commit.outputs.sha_short }} ${{ steps.ref_name.outputs.ref_name && format('{0}:{1}', inputs.registry, steps.ref_name.outputs.ref_name) || '' }} ${{ inputs.latest && format('{0}:{1}', inputs.registry, 'latest') || '' }} outputs: type=image,oci-mediatypes=true,compression=zstd,compression-level=3,force-compression=true,push=true target: ${{ inputs.target }} ```

For the logs, here is a logs for "Post Build Image" step which is hanging:

Details

``` ##[debug]Evaluating condition for step: 'Post Build Image' ##[debug]Evaluating: always() ##[debug]Evaluating always: ##[debug]=> true ##[debug]Result: true ##[debug]Starting: Post Build Image ##[debug]Loading inputs ##[debug]Evaluating: inputs.build-args ##[debug]Evaluating Index: ##[debug]..Evaluating inputs: ##[debug]..=> Object ##[debug]..Evaluating String: ##[debug]..=> 'build-args' ##[debug]=> '' ##[debug]Result: '' ##[debug]Evaluating: format('type=registry,ref={0}:latest ##[debug]{1} ##[debug]', inputs.registry, ((steps.ref_name.outputs.ref_name && format('type=registry,ref={0}:{1}', inputs.registry, steps.ref_name.outputs.ref_name)) || '')) ##[debug]Evaluating format: ##[debug]..Evaluating String: ##[debug]..=> 'type=registry,ref={0}:latest ##[debug]{1} ##[debug]' ##[debug]..Evaluating Index: ##[debug]....Evaluating inputs: ##[debug]....=> Object ##[debug]....Evaluating String: ##[debug]....=> 'registry' ##[debug]..=> '1234567890.dkr.ecr.us-east-1.amazonaws.com/app' ##[debug]..Evaluating Or: ##[debug]....Evaluating And: ##[debug]......Evaluating Index: ##[debug]........Evaluating Index: ##[debug]..........Evaluating Index: ##[debug]............Evaluating steps: ##[debug]............=> Object ##[debug]............Evaluating String: ##[debug]............=> 'ref_name' ##[debug]..........=> Object ##[debug]..........Evaluating String: ##[debug]..........=> 'outputs' ##[debug]........=> Object ##[debug]........Evaluating String: ##[debug]........=> 'ref_name' ##[debug]......=> 'kk-testing-workflow' ##[debug]......Evaluating format: ##[debug]........Evaluating String: ##[debug]........=> 'type=registry,ref={0}:{1}' ##[debug]........Evaluating Index: ##[debug]..........Evaluating inputs: ##[debug]org.opencontainers.image.source=github.com/{1} ##[debug]{2} ##[debug]' ##[debug]..Evaluating Index: ##[debug]....Evaluating github: ##[debug]....=> Object ##[debug]....Evaluating String: ##[debug]....=> 'sha' ##[debug]..=> '7aff86b34883cef456bee9f0f21e58012b6391a7' ##[debug]..Evaluating Index: ##[debug]....Evaluating github: ##[debug]....=> Object ##[debug]....Evaluating String: ##[debug]....=> 'repository' ##[debug]..=> '%%%ORG%%/%%%REPO%%' ##[debug]..Evaluating Index: ##[debug]....Evaluating inputs: ##[debug]....=> Object ##[debug]....Evaluating String: ##[debug]....=> 'labels' ##[debug]..=> '' ##[debug]=> 'org.opencontainers.image.revision=7aff86b34883cef456bee9f0f21e58012b6391a7 ##[debug]org.opencontainers.image.source=github.com/%%%ORG%%/%%%REPO%% ##[debug] ##[debug]' ##[debug]Result: 'org.opencontainers.image.revision=7aff86b34883cef456bee9f0f21e58012b6391a7 ##[debug]org.opencontainers.image.source=github.com/%%%ORG%%/%%%REPO%% ##[debug] ##[debug]' ##[debug]Evaluating: inputs.platforms ##[debug]Evaluating Index: ##[debug]..Evaluating inputs: ##[debug]..=> Object ##[debug]..Evaluating String: ##[debug]..=> 'platforms' ##[debug]=> 'linux/arm64 ##[debug]' ##[debug]Result: 'linux/arm64 ##[debug]' ##[debug]Evaluating: secrets.build-secrets ##[debug]Evaluating Index: ##[debug]..Evaluating secrets: ##[debug]..=> Object ##[debug]..Evaluating String: ##[debug]..=> 'build-secrets' ##[debug]=> '*** ##[debug]' ##[debug]Result: '*** ##[debug]' ##[debug]Evaluating: format('{0}:{1} ##[debug]{2} ##[debug]{3} ##[debug]', inputs.registry, steps.get_commit.outputs.sha_short, ((steps.ref_name.outputs.ref_name && format('{0}:{1}', inputs.registry, steps.ref_name.outputs.ref_name)) || ''), ((inputs.latest && format('{0}:{1}', inputs.registry, 'latest')) || '')) ##[debug]Evaluating format: ##[debug]..Evaluating String: ##[debug]..=> '{0}:{1} ##[debug]{2} ##[debug]{3} ##[debug]' ##[debug]..Evaluating Index: ##[debug]....Evaluating inputs: ##[debug]....=> Object ##[debug]....Evaluating String: ##[debug]....=> 'registry' ##[debug]..=> '1234567890.dkr.ecr.us-east-1.amazonaws.com/app' ##[debug]..Evaluating Index: ##[debug]....Evaluating Index: ##[debug]......Evaluating Index: ##[debug]........Evaluating steps: ##[debug]........=> Object ##[debug]........Evaluating String: ##[debug]........=> 'get_commit' ##[debug]......=> Object ##[debug]......Evaluating String: ##[debug]......=> 'outputs' ##[debug]....=> Object ##[debug]....Evaluating String: ##[debug]....=> 'sha_short' ##[debug]..=> '7aff86b' ##[debug]..Evaluating Or: ##[debug]....Evaluating And: ##[debug]......Evaluating Index: ##[debug]........Evaluating Index: ##[debug]..........Evaluating Index: ##[debug]............Evaluating steps: ##[debug]............=> Object ##[debug]............Evaluating String: ##[debug]............=> 'ref_name' ##[debug]..........=> Object ##[debug]..........Evaluating String: ##[debug]..........=> 'outputs' ##[debug]........=> Object ##[debug]........Evaluating String: ##[debug]........=> 'ref_name' ##[debug]......=> 'kk-testing-workflow' ##[debug]......Evaluating format: ##[debug]........Evaluating String: ##[debug]........=> '{0}:{1}' ##[debug]........Evaluating Index: ##[debug]..........Evaluating inputs: ##[debug]..........=> Object ##[debug]..........Evaluating String: ##[debug]..........=> 'registry' ##[debug]........=> '1234567890.dkr.ecr.us-east-1.amazonaws.com/app' ##[debug]........Evaluating Index: ##[debug]..........Evaluating Index: ##[debug]............Evaluating Index: ##[debug]..............Evaluating steps: ##[debug]..............=> Object ##[debug]..............Evaluating String: ##[debug]..............=> 'ref_name' ##[debug]............=> Object ##[debug]............Evaluating String: ##[debug]............=> 'outputs' ##[debug]..........=> Object ##[debug]..........Evaluating String: ##[debug]..........=> 'ref_name' ##[debug]........=> 'kk-testing-workflow' ##[debug]......=> '1234567890.dkr.ecr.us-east-1.amazonaws.com/app:kk-testing-workflow' ##[debug]....=> '1234567890.dkr.ecr.us-east-1.amazonaws.com/app:kk-testing-workflow' ##[debug]..=> '1234567890.dkr.ecr.us-east-1.amazonaws.com/app:kk-testing-workflow' ##[debug]..Evaluating Or: ##[debug]....Evaluating And: ##[debug]......Evaluating Index: ##[debug]........Evaluating inputs: ##[debug]........=> Object ##[debug]........Evaluating String: ##[debug]........=> 'latest' ##[debug]......=> false ##[debug]....=> false ##[debug]....Evaluating String: ##[debug]....=> '' ##[debug]..=> '' ##[debug]=> '1234567890.dkr.ecr.us-east-1.amazonaws.com/app:7aff86b ##[debug]1234567890.dkr.ecr.us-east-1.amazonaws.com/app:kk-testing-workflow ##[debug] ##[debug]' ##[debug]Result: '1234567890.dkr.ecr.us-east-1.amazonaws.com/app:7aff86b ##[debug]1234567890.dkr.ecr.us-east-1.amazonaws.com/app:kk-testing-workflow ##[debug] ##[debug]' ##[debug]Evaluating: inputs.target ##[debug]Evaluating Index: ##[debug]..Evaluating inputs: ##[debug]..=> Object ##[debug]..Evaluating String: ##[debug]..=> 'target' ##[debug]=> '' ##[debug]Result: '' ##[debug]Evaluating: github.token ##[debug]Evaluating Index: ##[debug]..Evaluating github: ##[debug]..=> Object ##[debug]..Evaluating String: ##[debug]..=> 'token' ##[debug]=> '***' ##[debug]Result: '***' ##[debug]Loading env Post job cleanup. ::group::Generating build summary Generating build summary ##[debug]Docker.isAvailable ok: /usr/bin/docker ##[debug]Docker.isAvailable ok: /usr/bin/docker ##[debug]Buildx.isStandalone: false ##[debug]Exec.getExecOutput: docker buildx version ##[debug]Buildx.versionSatisfies 0.15.1 statisfies >=0.13.0: true exporting build record to /home/runner/work/_temp/docker-actions-toolkit-YdNBNR/export ##[debug]Exec.exec: mkfifo /home/runner/work/_temp/docker-actions-toolkit-YdNBNR/buildx-in-QT8MUl.fifo /usr/bin/mkfifo /home/runner/work/_temp/docker-actions-toolkit-YdNBNR/buildx-in-QT8MUl.fifo ##[debug]Exec.exec: mkfifo /home/runner/work/_temp/docker-actions-toolkit-YdNBNR/buildx-out-NQb0Ke.fifo /usr/bin/mkfifo /home/runner/work/_temp/docker-actions-toolkit-YdNBNR/buildx-out-NQb0Ke.fifo ##[debug]Docker.isAvailable ok: /usr/bin/docker ##[debug]Buildx.isStandalone: false docker buildx --builder builder-35557753-3571-48a9-a44e-155aaf174933 dial-stdio docker run --rm -i -v /home/runner/.docker/buildx/refs:/buildx-refs -v /home/runner/work/_temp/docker-actions-toolkit-YdNBNR/export:/out docker.io/dockereng/export-build:latest --ref-state-dir=/buildx-refs --node=builder-35557753-3571-48a9-a44e-155aaf174933/builder-35557753-3571-48a9-a44e-155aaf1749330 --ref=sc9402epeyflghe5p2douqpvx --uid=1001 --gid=127 Unable to find image 'dockereng/export-build:latest' locally latest: Pulling from dockereng/export-build bb431e31f450: Pulling fs layer 03f1761d02d6: Pulling fs layer 03f1761d02d6: Verifying Checksum 03f1761d02d6: Download complete bb431e31f450: Verifying Checksum bb431e31f450: Download complete bb431e31f450: Pull complete 03f1761d02d6: Pull complete Digest: sha256:3dfedea3148487c108965dede834f22e81528fc5b2f3989e4b8ecec2f8fe10ae Status: Downloaded newer image for dockereng/export-build:latest Parsing /home/runner/work/_temp/docker-actions-toolkit-YdNBNR/export/summary.json Build record exported to /home/runner/work/_temp/docker-actions-toolkit-YdNBNR/export/%%%ORG%%~%%%REPO%%~SC9402.dockerbuild (19.37 KB) ##[debug]Workflow Run Backend ID: 881487f2-9121-40b6-b280-0fc26a36f364 ##[debug]Workflow Job Run Backend ID: 2ccbf1db-6c73-5af8-5ba2-cf05469dcf51 Uploading %%%ORG%%~%%%REPO%%~SC9402.dockerbuild to blob storage ##[debug][Request] CreateArtifact https://results-receiver.actions.githubusercontent.com/twirp/github.actions.results.api.v1.ArtifactService/CreateArtifact ##[debug][Response] - 200 ##[debug]Headers: { ##[debug] "content-length": "583", ##[debug] "content-type": "application/json", ##[debug] "date": "Thu, 27 Jun 2024 22:37:29 GMT", ##[debug] "x-github-backend": "Kubernetes", ##[debug] "x-github-request-id": "BCD3:19861D:1E7943B:29104EF:667DE9A9" ##[debug]} ##[debug]Body: { ##[debug] "ok": true, ##[debug] "signed_upload_url": "https://productionresultssa8.blob.core.windows.net/actions-results/881487f2-9121-40b6-b280-0fc26a36f364/workflow-job-run-2ccbf1db-6c73-5af8-5ba2-cf05469dcf51/artifacts/50568a2c63c0f6bf0bd743788f352a88422f9f805433b0b4b83ca3f1d969f6c4.zip?se=2024-06-27T23%3A37%3A29Z&sig=9%2FFxKc4Ob9majqStd4%2FtEZatovCdz%2Fmv5P3n71pNDQE%3D&ske=2024-06-28T06%3A58%3A07Z&skoid=ca7593d4-ee42-46cd-af88-8b886a2f84eb&sks=b&skt=2024-06-27T18%3A58%3A07Z&sktid=398a6654-997b-47e9-b12b-9515b896b4de&skv=2023-11-03&sp=cw&spr=https&sr=b&st=2024-06-27T22%3A37%3A24Z&sv=2023-11-03" ##[debug]} ##[debug]Upload headers: {"blobContentDisposition":"attachment; filename=\"%%%ORG%%~%%%REPO%%~SC9402.dockerbuild\"","blobContentType":"application/gzip"} Beginning upload of artifact content to blob storage Uploaded bytes 19835 Finished uploading artifact content to blob storage! SHA256 hash of uploaded artifact is 251c716c56bca567094d08332bfa354e28c82136d40d98bbae8d51c23cfaffe9 Finalizing artifact upload ##[debug][Request] FinalizeArtifact https://results-receiver.actions.githubusercontent.com/twirp/github.actions.results.api.v1.ArtifactService/FinalizeArtifact ##[debug][Response] - 200 ##[debug]Headers: { ##[debug] "content-length": "38", ##[debug] "content-type": "application/json", ##[debug] "date": "Thu, 27 Jun 2024 22:37:30 GMT", ##[debug] "x-github-backend": "Kubernetes", ##[debug] "x-github-request-id": "BCD4:2435A7:1EAF2B1:294627F:667DE9A9" ##[debug]} ##[debug]Body: { ##[debug] "ok": true, ##[debug] "artifact_id": "1646658588" ##[debug]} Artifact successfully finalized (1646658588) Artifact download URL: https://github.com/%%%ORG%%/%%%REPO%%/actions/runs/9704407165/artifacts/1646658588 Writing summary ::endgroup:: ::group::Removing temp folder /home/runner/work/_temp/docker-actions-toolkit-ht4wrg Removing temp folder /home/runner/work/_temp/docker-actions-toolkit-ht4wrg ::endgroup:: ::group::Post cache Post cache State not set ::endgroup:: ##[debug]Re-evaluate condition on job cancellation for step: 'Post Build Image'. ```

I noticed it references builder, so here is a snippet of setup-buildx output:

Details

``` 2024-06-27T22:37:23.1893506Z ##[debug]Set output name = builder-35557753-3571-48a9-a44e-155aaf174933 2024-06-27T22:37:23.1894456Z ##[debug]Set output driver = kubernetes 2024-06-27T22:37:23.1895076Z ##[debug]Set output platforms = linux/arm64 2024-06-27T22:37:23.1895721Z ##[debug]Set output nodes = [ 2024-06-27T22:37:23.1896203Z ##[debug] { 2024-06-27T22:37:23.1896689Z ##[debug] "name": "builder-35557753-3571-48a9-a44e-155aaf1749330", 2024-06-27T22:37:23.1897771Z ##[debug] "endpoint": "kubernetes:///builder-35557753-3571-48a9-a44e-155aaf174933?deployment=buildkit-df771ebf-a24e-48a7-b204-4c0aceb2444b-f59bb&kubeconfig=", 2024-06-27T22:37:23.1898805Z ##[debug] "driver-opts": [ 2024-06-27T22:37:23.1899231Z ##[debug] "namespace=buildkit", 2024-06-27T22:37:23.1899813Z ##[debug] "nodeselector=kubernetes.io/arch=arm64" 2024-06-27T22:37:23.1900336Z ##[debug] ], 2024-06-27T22:37:23.1900711Z ##[debug] "status": "running", 2024-06-27T22:37:23.1901560Z ##[debug] "buildkitd-flags": "--allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host", 2024-06-27T22:37:23.1902383Z ##[debug] "buildkit": "v0.14.1", 2024-06-27T22:37:23.1902886Z ##[debug] "platforms": "linux/arm64", 2024-06-27T22:37:23.1903326Z ##[debug] "features": { 2024-06-27T22:37:23.1904116Z ##[debug] "Automatically load images to the Docker Engine image store": true, 2024-06-27T22:37:23.1904884Z ##[debug] "Cache export": true, 2024-06-27T22:37:23.1905360Z ##[debug] "Docker exporter": true, 2024-06-27T22:37:23.1905902Z ##[debug] "Multi-platform build": true, 2024-06-27T22:37:23.1906390Z ##[debug] "OCI exporter": true 2024-06-27T22:37:23.1906801Z ##[debug] }, 2024-06-27T22:37:23.1907266Z ##[debug] "labels": { 2024-06-27T22:37:23.1907764Z ##[debug] "org.mobyproject.buildkit.worker.executor": "oci", 2024-06-27T22:37:23.1908731Z ##[debug] "org.mobyproject.buildkit.worker.hostname": "builder-35557753-3571-48a9-a44e-155aaf1749330-5b88b9976f-sb42h", 2024-06-27T22:37:23.1909646Z ##[debug] "org.mobyproject.buildkit.worker.network": "host", 2024-06-27T22:37:23.1910348Z ##[debug] "org.mobyproject.buildkit.worker.oci.process-mode": "sandbox", 2024-06-27T22:37:23.1911228Z ##[debug] "org.mobyproject.buildkit.worker.selinux.enabled": "false", 2024-06-27T22:37:23.1911951Z ##[debug] "org.mobyproject.buildkit.worker.snapshotter": "overlayfs" 2024-06-27T22:37:23.1912605Z ##[debug] }, 2024-06-27T22:37:23.1912955Z ##[debug] "gcPolicy": [ 2024-06-27T22:37:23.1913328Z ##[debug] { 2024-06-27T22:37:23.1913763Z ##[debug] "all": false, 2024-06-27T22:37:23.1914200Z ##[debug] "filter": [ 2024-06-27T22:37:23.1914694Z ##[debug] "type==source.local", 2024-06-27T22:37:23.1915170Z ##[debug] "type==exec.cachemount", 2024-06-27T22:37:23.1915661Z ##[debug] "type==source.git.checkout" 2024-06-27T22:37:23.1916234Z ##[debug] ], 2024-06-27T22:37:23.1916616Z ##[debug] "keepDuration": "48h0m0s", 2024-06-27T22:37:23.1917374Z ##[debug] "keepBytes": "488.3MiB" 2024-06-27T22:37:23.1917802Z ##[debug] }, 2024-06-27T22:37:23.1918159Z ##[debug] { 2024-06-27T22:37:23.1918573Z ##[debug] "all": false, 2024-06-27T22:37:23.1919021Z ##[debug] "keepDuration": "1440h0m0s", 2024-06-27T22:37:23.1919493Z ##[debug] "keepBytes": "12.11GiB" 2024-06-27T22:37:23.1920017Z ##[debug] }, 2024-06-27T22:37:23.1920385Z ##[debug] { 2024-06-27T22:37:23.1920793Z ##[debug] "all": false, 2024-06-27T22:37:23.1921220Z ##[debug] "keepBytes": "12.11GiB" 2024-06-27T22:37:23.1923938Z ##[debug] }, 2024-06-27T22:37:23.1924447Z ##[debug] { 2024-06-27T22:37:23.1924834Z ##[debug] "all": true, 2024-06-27T22:37:23.1925362Z ##[debug] "keepBytes": "12.11GiB" 2024-06-27T22:37:23.1925815Z ##[debug] } 2024-06-27T22:37:23.1926153Z ##[debug] ] 2024-06-27T22:37:23.1926570Z ##[debug] } 2024-06-27T22:37:23.1926913Z ##[debug]] 2024-06-27T22:37:23.1927944Z ##[debug]Set output endpoint = kubernetes:///builder-35557753-3571-48a9-a44e-155aaf174933?deployment=buildkit-df771ebf-a24e-48a7-b204-4c0aceb2444b-f59bb&kubeconfig= 2024-06-27T22:37:23.1928986Z ##[debug]Set output status = running 2024-06-27T22:37:23.1929786Z ##[debug]Set output flags = --allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host 2024-06-27T22:37:23.1931043Z ##[debug]Finishing: Set up Docker Buildx ```

After job cancellation, builder pods in kubernetes keep running. Their logs have a few of these (might be red herring)

time="2024-06-27T22:37:27Z" level=warning msg="encountered unknown type application/vnd.oci.image.layer.v1.tar+zstd; children may not be fetched" span="exporting to image"
time="2024-06-27T22:37:27Z" level=warning msg="encountered unknown type application/vnd.oci.image.layer.v1.tar+zstd; children may not be fetched" span="exporting to image"

also this under workflow summary:

Build Image / Build and Push Image
Runner GitHub Actions 69 did not respond to a cancelation request with 00:05:00.

crazy-max commented 4 days ago

@kkopachev Thanks!

using kubernetes driver.

Ok you're using the kubernetes driver but don't think there is a dial issue when exporting the build in "Post Build Image"

  Beginning upload of artifact content to blob storage
  Uploaded bytes 19835
  Finished uploading artifact content to blob storage!
  SHA256 hash of uploaded artifact is 251c716c56bca567094d08332bfa354e28c82136d40d98bbae8d51c23cfaffe9
  Finalizing artifact upload
  ##[debug][Request] FinalizeArtifact https://results-receiver.actions.githubusercontent.com/twirp/github.actions.results.api.v1.ArtifactService/FinalizeArtifact
  ##[debug][Response] - 200
  ##[debug]Headers: {
  ##[debug]  "content-length": "38",
  ##[debug]  "content-type": "application/json",
  ##[debug]  "date": "Thu, 27 Jun 2024 22:37:30 GMT",
  ##[debug]  "x-github-backend": "Kubernetes",
  ##[debug]  "x-github-request-id": "BCD4:2435A7:1EAF2B1:294627F:667DE9A9"
  ##[debug]}
  ##[debug]Body: {
  ##[debug]  "ok": true,
  ##[debug]  "artifact_id": "1646658588"
  ##[debug]}
  Artifact successfully finalized (1646658588)
  Artifact download URL: https://github.com/%%%ORG%%/%%%REPO%%/actions/runs/9704407165/artifacts/1646658588
  Writing summary

Looking at your logs it seems to write the summary so does it hangs right after "Writing summary" or is it just printed after cancellation but hangs before that?

kkopachev commented 4 days ago

"Writing summary" is where it hangs. I can see artifact uploaded successfully. I guess it's something in the summary that break it. If you could dump raw unformatted summary (json?) to a file and upload it as artifact before writing summary, I can rerun my workflow to hopefully see if there are anything suspicious. For me it looks like an issue with summary has something that breaks the worker

crazy-max commented 3 days ago

"Writing summary" is where it hangs

Thanks that's useful, I think the process exporting the build record remains opened which make this post step hangs. Probably around https://github.com/docker/actions-toolkit/blob/fe9937dd36b64d2090fbfec40e144944ae390a12/src/buildx/history.ts#L115. I will take a closer look.

crazy-max commented 3 days ago

@kkopachev I made some changes in https://github.com/docker/actions-toolkit/pull/392

Can you try with:

uses: crazy-max/docker-build-push-action@test-process-term

And give me the logs of the post step please? Thanks!

kkopachev commented 2 days ago

@crazy-max nothing changed, I don't see new logs. I think you'd have to re-generate dist.js as this commit happened before this commit

If it is relevant, build image is all cached

crazy-max commented 2 days ago

nothing changed, I don't see new logs.

I have new logs on my side like https://github.com/crazy-max/docker-build-push-action/actions/runs/9730285429/job/26853184981#step:6:20:

kkopachev commented 1 day ago

oh, didn't notice that. Still hangs. line Process "docker run" exited with code 0 in there

Full logs

``` blah-blah, evaluating inputs, tokens, etc... Post job cleanup. ::group::Generating build summary Generating build summary ##[debug]Docker.isAvailable ok: /usr/bin/docker ##[debug]Docker.isAvailable ok: /usr/bin/docker ##[debug]Buildx.isStandalone: false ##[debug]Exec.getExecOutput: docker buildx version ##[debug]Buildx.versionSatisfies 0.15.1 statisfies >=0.13.0: true exporting build record to /home/runner/work/_temp/docker-actions-toolkit-T3xO7E/export ##[debug]Exec.exec: mkfifo /home/runner/work/_temp/docker-actions-toolkit-T3xO7E/buildx-in-JnpRvP.fifo /usr/bin/mkfifo /home/runner/work/_temp/docker-actions-toolkit-T3xO7E/buildx-in-JnpRvP.fifo ##[debug]Exec.exec: mkfifo /home/runner/work/_temp/docker-actions-toolkit-T3xO7E/buildx-out-jzAGEi.fifo /usr/bin/mkfifo /home/runner/work/_temp/docker-actions-toolkit-T3xO7E/buildx-out-jzAGEi.fifo ##[debug]Docker.isAvailable ok: /usr/bin/docker ##[debug]Buildx.isStandalone: false docker buildx --builder builder-ba1f2a46-f993-4bde-a25c-4f953e2b7e3c dial-stdio docker run --rm -i -v /home/runner/.docker/buildx/refs:/buildx-refs -v /home/runner/work/_temp/docker-actions-toolkit-T3xO7E/export:/out docker.io/dockereng/export-build:latest --ref-state-dir=/buildx-refs --node=builder-ba1f2a46-f993-4bde-a25c-4f953e2b7e3c/builder-ba1f2a46-f993-4bde-a25c-4f953e2b7e3c0 --ref=szp2slrbwwx7sgc4ueqqbytx6 --uid=1001 --gid=127 Unable to find image 'dockereng/export-build:latest' locally latest: Pulling from dockereng/export-build bb431e31f450: Pulling fs layer 03f1761d02d6: Pulling fs layer 03f1761d02d6: Download complete bb431e31f450: Verifying Checksum bb431e31f450: Download complete bb431e31f450: Pull complete 03f1761d02d6: Pull complete Digest: sha256:3dfedea3148487c108965dede834f22e81528fc5b2f3989e4b8ecec2f8fe10ae Status: Downloaded newer image for dockereng/export-build:latest Process "docker run" exited with code 0 ##[debug]Terminating "buildx dial-stdio" process Parsing /home/runner/work/_temp/docker-actions-toolkit-T3xO7E/export/summary.json Build record exported to /home/runner/work/_temp/docker-actions-toolkit-T3xO7E/export/%%%ORG%%%~%%%REPO%%%~SZP2SL.dockerbuild (19.29 KB) ##[debug]Workflow Run Backend ID: 41f1dabb-d45a-46bf-a9de-0b21d22a9f87 ##[debug]Workflow Job Run Backend ID: 2ccbf1db-6c73-5af8-5ba2-cf05469dcf51 Uploading %%%ORG%%%~%%%REPO%%%~SZP2SL.dockerbuild to blob storage ##[debug][Request] CreateArtifact https://results-receiver.actions.githubusercontent.com/twirp/github.actions.results.api.v1.ArtifactService/CreateArtifact ##[debug][Response] - 200 ##[debug]Headers: { ##[debug] "content-length": "579", ##[debug] "content-type": "application/json", ##[debug] "date": "Mon, 01 Jul 2024 04:37:19 GMT", ##[debug] "x-github-backend": "Kubernetes", ##[debug] "x-github-request-id": "D80B:D6B68:1ECE19:30119C:6682327F" ##[debug]} ##[debug]Body: { ##[debug] "ok": true, ##[debug] "signed_upload_url": "https://productionresultssa7.blob.core.windows.net/actions-results/41f1dabb-d45a-46bf-a9de-0b21d22a9f87/workflow-job-run-2ccbf1db-6c73-5af8-5ba2-cf05469dcf51/artifacts/abed9589a335347e71d127873d588c94b0ed2ab380bfedb7455f9ae244ab10a9.zip?se=2024-07-01T05%3A37%3A19Z&sig=gLnooCiHKjughziQruzVI1UZu%2BBVy53ZGnqZgqrRk5c%3D&ske=2024-07-01T14%3A52%3A10Z&skoid=ca7593d4-ee42-46cd-af88-8b886a2f84eb&sks=b&skt=2024-07-01T02%3A52%3A10Z&sktid=398a6654-997b-47e9-b12b-9515b896b4de&skv=2023-11-03&sp=cw&spr=https&sr=b&st=2024-07-01T04%3A37%3A14Z&sv=2023-11-03" ##[debug]} ##[debug]Upload headers: {"blobContentDisposition":"attachment; filename=\"%%%ORG%%%~%%%REPO%%%~SZP2SL.dockerbuild\"","blobContentType":"application/gzip"} Beginning upload of artifact content to blob storage Uploaded bytes 19756 Finished uploading artifact content to blob storage! SHA256 hash of uploaded artifact is 6e8fb00c2005c2f62d0b7fb5846d74696e856501f9e4d5e4e24267ff240f6579 Finalizing artifact upload ##[debug][Request] FinalizeArtifact https://results-receiver.actions.githubusercontent.com/twirp/github.actions.results.api.v1.ArtifactService/FinalizeArtifact ##[debug][Response] - 200 ##[debug]Headers: { ##[debug] "content-length": "38", ##[debug] "content-type": "application/json", ##[debug] "date": "Mon, 01 Jul 2024 04:37:19 GMT", ##[debug] "x-github-backend": "Kubernetes", ##[debug] "x-github-request-id": "D80C:244133:1DFC86:2F2F2A:6682327F" ##[debug]} ##[debug]Body: { ##[debug] "ok": true, ##[debug] "artifact_id": "1653888073" ##[debug]} Artifact successfully finalized (1653888073) Artifact download URL: https://github.com/%%%ORG%%%/%%%REPO%%%/actions/runs/9729523617/artifacts/1653888073 Writing summary ::endgroup:: ::group::Removing temp folder /home/runner/work/_temp/docker-actions-toolkit-78OqIp Removing temp folder /home/runner/work/_temp/docker-actions-toolkit-78OqIp ::group::Post cache Post cache ```

crazy-max commented 1 day ago

##[debug]Terminating "buildx dial-stdio" process

Ok looks like this process is the culprit, seems https://github.com/docker/actions-toolkit/pull/392/files#diff-fcf6a886ba5fdffc4fb064e49c7776a41c5fa00be72b7aef6f56305b8632ad8aR175 has no effect and process is not gracefully terminated. Will take a closer look, thanks for testing this out!

Edit: made new changes if you can try again with:

uses: crazy-max/docker-build-push-action@test-process-term

Thanks!

crazy-max commented 10 hours ago

You can test with latest changes on default branch:

uses: docker/build-push-action@master

kkopachev commented 9 hours ago

@crazy-max It still hangs for me

Post job cleanup.
Generating build summary
  exporting build record to /home/runner/work/_temp/docker-actions-toolkit-uudZge/export
  /usr/bin/mkfifo /home/runner/work/_temp/docker-actions-toolkit-uudZge/buildx-in-cGEj9n.fifo
  /usr/bin/mkfifo /home/runner/work/_temp/docker-actions-toolkit-uudZge/buildx-out-q511v1.fifo
  docker buildx --builder builder-c5c24b3c-0fa7-47c6-aa2c-051417efaec2 dial-stdio
  docker run --rm -i -v /home/runner/.docker/buildx/refs:/buildx-refs -v /home/runner/work/_temp/docker-actions-toolkit-uudZge/export:/out docker.io/dockereng/export-build:latest --ref-state-dir=/buildx-refs --node=builder-c5c24b3c-0fa7-47c6-aa2c-051417efaec2/builder-c5c24b3c-0fa7-47c6-aa2c-051417efaec20 --ref=s75b1bml5c3fwe58crd0uh01l --uid=1001 --gid=127
  Unable to find image 'dockereng/export-build:latest' locally
  latest: Pulling from dockereng/export-build
  bb431e31f450: Pulling fs layer
  03f1761d02d6: Pulling fs layer
  03f1761d02d6: Verifying Checksum
  03f1761d02d6: Download complete
  bb431e31f450: Verifying Checksum
  bb431e31f450: Download complete
  bb431e31f450: Pull complete
  03f1761d02d6: Pull complete
  Digest: sha256:3dfedea3148487c108965dede834f22e81528fc5b2f3989e4b8ecec2f8fe10ae
  Status: Downloaded newer image for dockereng/export-build:latest
  Process "docker run" exited with code 0
  Parsing /home/runner/work/_temp/docker-actions-toolkit-uudZge/export/summary.json
  Build record exported to /home/runner/work/_temp/docker-actions-toolkit-uudZge/export/ORG~REPO~S75B1B.dockerbuild (19.36 KB)
  Uploading ORG~REPO~S75B1B.dockerbuild to blob storage
  Process "buildx dial-stdio" was killed with signal SIGKILL
  Beginning upload of artifact content to blob storage
  Uploaded bytes 19829
  Finished uploading artifact content to blob storage!
  SHA256 hash of uploaded artifact is 27f8ee69f5b57b62210e20cd4be08cd97501b27518c8aa36a370093281585d74
  Finalizing artifact upload
  Artifact successfully finalized (1660322982)
  Artifact download URL: https://github.com/ORG/REPO/actions/runs/9764057304/artifacts/1660322982
  Writing summary
Removing temp folder /home/runner/work/_temp/docker-actions-toolkit-3Hcbvk
Post cache
  State not set

crazy-max commented 8 hours ago

@crazy-max It still hangs for me

Ok then it's not related to child processes as I see both exit in the post step from your logs. Will take a closer look by using the kubernetes driver but for now consider disabling summary as shown in https://github.com/docker/build-push-action/issues/1156#issuecomment-2194160238 if this disrupts your workflow.

docker / build-push-action