search

docker / buildx

Docker CLI plugin for extended build capabilities with BuildKit
Apache License 2.0
3.42k stars 465 forks source link

`COPY --link --chown=user` (Fails with "failed to solve: invalid user index: -1") #1526

Closed polarathene closed 1 year ago

polarathene commented 1 year ago

UPDATE 2: Resolved. This was a misunderstanding on my part. I didn't realize Docker Engine 20.10.22 from Dec 2022 still used BuildKit v0.8.4 from Apr 2021. buildx v0.9.1 was using BuildKit v0.11.0 instead where --link is actually supported not silently ignored for backwards compatibility.

UPDATE: See next message for a minimal reproduction. Not a cache issue, seems specific to buildx when using the docker-container driver.

I also somehow missed an existing issue with the same error message.

Original report (invalid) ~~I am assuming this has something to do with caching we have used in CI (_via the Github Action `docker-build-push`_), as a local build does not fail like this.~~ ## Local This correctly uses existing layer cache up until the change for `COPY --link` adding `--chown=clamav`. Command run: ```bash DOCKER_BUILDKIT=1 docker build \ --tag 'mailserver-testing:ci' \ --build-arg VCS_VERSION=$(shell git rev-parse --short HEAD) \ --build-arg VCS_REVISION=$(shell cat VERSION) \ . ```
Terminal output ``` [+] Building 45.8s (60/60) FINISHED => [internal] load build definition from Dockerfile 0.1s => => transferring dockerfile: 12.09kB 0.0s => [internal] load .dockerignore 0.2s => => transferring context: 34B 0.0s => resolve image config for docker.io/docker/dockerfile:1 1.7s => CACHED docker-image://docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81 0.0s => [internal] load .dockerignore 0.0s => [internal] load build definition from Dockerfile 0.0s => [internal] load metadata for docker.io/clamav/clamav:latest 0.0s => [internal] load metadata for docker.io/library/debian:11-slim 0.0s => [stage-base 1/49] FROM docker.io/library/debian:11-slim 0.0s => [internal] load build context 0.1s => => transferring context: 25.72kB 0.0s => CACHED FROM docker.io/clamav/clamav:latest 0.0s => CACHED [stage-base 2/49] COPY target/bin/sedfile /usr/local/bin/sedfile 0.0s => CACHED [stage-base 3/49] RUN chmod +x /usr/local/bin/sedfile 0.0s => CACHED [stage-base 4/49] COPY target/scripts/build/* /build/ 0.0s => CACHED [stage-base 5/49] COPY target/scripts/helpers/log.sh /usr/local/bin/helpers/log.sh 0.0s => CACHED [stage-base 6/49] RUN /bin/bash /build/packages.sh 0.0s => CACHED [stage-base 7/49] RUN </etc/cron.d/clamav-fr 0.0s => [stage-base 8/49] COPY --chown=clamav --link --from=docker.io/clamav/clamav:latest /var/lib/clamav /var/lib/cla 0.3s => [stage-base 9/49] COPY target/dovecot/auth-passwdfile.inc target/dovecot/auth-master.inc target/dovecot/??-*.co 0.3s => [stage-base 10/49] COPY target/dovecot/sieve/ /etc/dovecot/sieve/ 0.3s => [stage-base 11/49] COPY target/dovecot/dovecot-purge.cron /etc/cron.d/dovecot-purge.disabled 0.3s => [stage-base 12/49] RUN chmod 0 /etc/cron.d/dovecot-purge.disabled 0.7s => [stage-base 13/49] WORKDIR /usr/share/dovecot 0.3s => [stage-base 14/49] RUN < [stage-base 15/49] COPY target/rspamd/local.d/ /etc/rspamd/local.d/ 0.4s => [stage-base 16/49] COPY target/dovecot/dovecot-ldap.conf.ext /etc/dovecot 0.3s => [stage-base 17/49] COPY target/postfix/ldap-users.cf target/postfix/ldap-groups.cf target/postfix/ldap-ali 0.4s => [stage-base 18/49] RUN < [stage-base 19/49] COPY target/postsrsd/postsrsd /etc/default/postsrsd 0.7s => [stage-base 20/49] COPY target/postgrey/postgrey /etc/default/postgrey 0.3s => [stage-base 21/49] COPY target/postgrey/postgrey.init /etc/init.d/postgrey 0.3s => [stage-base 22/49] RUN < [stage-base 23/49] COPY target/amavis/conf.d/* /etc/amavis/conf.d/ 0.4s => [stage-base 24/49] RUN < [stage-base 25/49] RUN < [stage-base 26/49] COPY target/fail2ban/jail.local /etc/fail2ban/jail.local 0.7s => [stage-base 27/49] COPY target/fail2ban/fail2ban.d/fixes.local /etc/fail2ban/fail2ban.d/fixes.local 0.3s => [stage-base 28/49] RUN < [stage-base 29/49] COPY target/opendkim/opendkim.conf /etc/opendkim.conf 0.4s => [stage-base 30/49] COPY target/opendkim/default-opendkim /etc/default/opendkim 0.3s => [stage-base 31/49] COPY target/opendmarc/opendmarc.conf /etc/opendmarc.conf 0.3s => [stage-base 32/49] COPY target/opendmarc/default-opendmarc /etc/default/opendmarc 0.3s => [stage-base 33/49] COPY target/opendmarc/ignore.hosts /etc/opendmarc/ignore.hosts 0.3s => [stage-base 34/49] RUN echo 'Reason_Message = Message {rejectdefer} due to: {spf}.' >>/etc/postfix-policyd-spf-p 0.7s => [stage-base 35/49] COPY target/fetchmail/fetchmailrc /etc/fetchmailrc_general 0.4s => [stage-base 36/49] COPY target/postfix/main.cf target/postfix/master.cf /etc/postfix/ 0.3s => [stage-base 37/49] COPY target/shared/ffdhe4096.pem /etc/postfix/dhparams.pem 0.7s => [stage-base 38/49] COPY target/shared/ffdhe4096.pem /etc/dovecot/dh.pem 0.2s => [stage-base 39/49] COPY target/postfix/header_checks.pcre target/postfix/sender_header_filter.pcre target/ 0.4s => [stage-base 40/49] RUN </etc/aliases...) 0.8s => [stage-base 41/49] RUN < [stage-base 42/49] COPY target/logwatch/maillog.conf /etc/logwatch/conf/logfiles/maillog.conf 0.4s => [stage-base 43/49] COPY target/supervisor/supervisord.conf /etc/supervisor/supervisord.conf 0.3s => [stage-base 44/49] COPY target/supervisor/conf.d/* /etc/supervisor/conf.d/ 0.3s => [stage-base 45/49] RUN < [stage-base 46/49] COPY VERSION / 0.7s => [stage-base 47/49] COPY target/bin/* target/scripts/*.sh target/scripts/startup/*.sh target/scripts/wrap 0.5s => [stage-base 48/49] RUN chmod +x /usr/local/bin/* 0.7s => [stage-base 49/49] COPY target/scripts/helpers /usr/local/bin/helpers 0.5s => exporting to image 11.3s => => exporting layers 11.2s => => writing image sha256:081f84dd66254a3515b331a5f0b24497fa3c67f1a01056119275f502d58fc344 0.0s => => naming to docker.io/library/mailserver-testing:ci ```
## CI This is what the command in CI looks like: ```bash /usr/bin/docker buildx build \ --cache-from type=local,src=/tmp/.buildx-cache \ --cache-to type=local,dest=/tmp/.buildx-cache-new,mode=max \ --iidfile /tmp/docker-build-push-r1dOWM/iidfile \ --output type=cacheonly \ --platform linux/amd64 \ --metadata-file /tmp/docker-build-push-r1dOWM/metadata-file \ . ``` ### CI logs The layer cache seems to be correctly used here too. I'm not sure why it fails other than the caching strategy likely being different? The `clamav` user does exist by this point (_created in an earlier `RUN` to a shell script that installs a package that adds the user_). Fails with the following error: ``` Error: buildx failed with: ERROR: failed to solve: invalid user index: -1 ```
Full build log ``` /usr/bin/docker buildx build --cache-from type=local,src=/tmp/.buildx-cache --cache-to type=local,dest=/tmp/.buildx-cache-new,mode=max --iidfile /tmp/docker-build-push-r1dOWM/iidfile --output type=cacheonly --platform linux/amd64 --metadata-file /tmp/docker-build-push-r1dOWM/metadata-file . #1 [internal] load .dockerignore #1 transferring context: 59B done #1 DONE 0.0s #2 [internal] load build definition from Dockerfile #2 transferring dockerfile: 12.09kB done #2 DONE 0.0s #3 resolve image config for docker.io/docker/dockerfile:1 #3 ... #4 [auth] docker/dockerfile:pull token for registry-1.docker.io #4 DONE 0.0s #3 resolve image config for docker.io/docker/dockerfile:1 #3 DONE 0.5s #5 docker-image://docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81e24ef1a0dbc #5 resolve docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81e24ef1a0dbc 0.0s done #5 sha256:1328b32c40fca9bcf9d70d8eccb72eb873d1[124](https://github.com/docker-mailserver/docker-mailserver/actions/runs/3945342546/jobs/6752151872#step:7:128)d72dadce04db8badbe7b08546 9.94MB / 9.94MB 0.1s done #5 extracting sha256:1328b32c40fca9bcf9d70d8eccb72eb873d1124d72dadce04db8badbe7b08546 #5 extracting sha256:1328b32c40fca9bcf9d70d8eccb72eb873d1124d72dadce04db8badbe7b08546 0.2s done #5 DONE 0.3s #6 [auth] clamav/clamav:pull token for registry-1.docker.io #6 DONE 0.0s #7 [auth] library/debian:pull token for registry-1.docker.io #7 DONE 0.0s #8 [internal] load metadata for docker.io/library/debian:11-slim #8 DONE 0.2s #9 [internal] load metadata for docker.io/clamav/clamav:latest #9 DONE 0.2s #10 [internal] load build context #10 DONE 0.0s #11 [stage-base 1/49] FROM docker.io/library/debian:11-slim@sha256:98d3b4b0cee264301eb1354e0b549323af2d0633e1c43375d0b25c01826b6790 #11 resolve docker.io/library/debian:11-slim@sha256:98d3b4b0cee264301eb1354e0b549323af2d0633e1c43375d0b25c01826b6790 0.0s done #11 DONE 0.0s #12 importing cache manifest from local:9656101879626942798 #12 DONE 0.0s #13 FROM docker.io/clamav/clamav:latest@sha256:314c46478306f1bbf3216e2a8ca4b3cb87ba5dd1e14fe4d43f0e3d13712a4af1 #13 resolve docker.io/clamav/clamav:latest@sha256:314c46478306f1bbf3216e2a8ca4b3cb87ba5dd1e14fe4d43f0e3d13712a4af1 #13 ... #10 [internal] load build context #10 transferring context: 295.46kB 0.0s done #10 DONE 0.1s #13 FROM docker.io/clamav/clamav:latest@sha256:314c46478306f1bbf3216e2a8ca4b3cb87ba5dd1e14fe4d43f0e3d13712a4af1 #13 resolve docker.io/clamav/clamav:latest@sha256:314c46478306f1bbf3216e2a8ca4b3cb87ba5dd1e14fe4d43f0e3d13712a4af1 0.0s done #13 sha256:24cb207b181c6103680fa838d6c19b2baa1869aee6afcd891ce46b7b174f7b26 1.26kB / 1.26kB 0.0s done #13 sha256:2d8c7c5ce39eaa53a3997135d3c16598f489f99655024514de17424d064b03e4 348B / 348B 0.0s done #13 sha256:68331520d622a7054a5e17fa38f2f5e25bf0680d93adfee677445f671ec4e3ba 1.66MB / 1.66MB 0.1s done #13 sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d1832dd2e0f2f8cf9e33 7.34MB / 231.14MB 0.2s #13 sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d1832dd2e0f2f8cf9e33 19.92MB / 231.14MB 0.3s #13 sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715 3.37MB / 3.37MB 0.1s done #13 sha256:cff7aab4088b8607efe477bdd4a9e0185bc8ed8339e171980897550e15390466 11.53MB / 78.00MB 0.2s #13 extracting sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715 #13 sha256:cff7aab4088b8607efe477bdd4a9e0185bc8ed8339e171980897550e15390466 30.41MB / 78.00MB 0.3s #13 extracting sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715 0.2s done #13 extracting sha256:68331520d622a7054a5e17fa38f2f5e25bf0680d93adfee677445f671ec4e3ba #13 sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d1832dd2e0f2f8cf9e33 34.83MB / 231.14MB 0.5s #13 sha256:cff7aab4088b8607efe477bdd4a9e0185bc8ed8339e171980897550e15390466 47.63MB / 78.00MB 0.5s #13 sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d1832dd2e0f2f8cf9e33 57.67MB / 231.14MB 0.6s #13 sha256:cff7aab4088b8607efe477bdd4a9e0185bc8ed8339e171980897550e15390466 59.77MB / 78.00MB 0.6s #13 sha256:cff7aab4088b8607efe477bdd4a9e0185bc8ed8339e171980897550e15390466 77.59MB / 78.00MB 0.8s #13 sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d1832dd2e0f2f8cf9e33 83.96MB / 231.14MB 0.9s #13 sha256:cff7aab4088b8607efe477bdd4a9e0185bc8ed8339e171980897550e15390466 78.00MB / 78.00MB 0.9s done #13 extracting sha256:68331520d622a7054a5e17fa38f2f5e25bf0680d93adfee677445f671ec4e3ba 0.5s done #13 extracting sha256:cff7aab4088b8607efe477bdd4a9e0185bc8ed8339e171980897550e15390466 #13 sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d1832dd2e0f2f8cf9e33 101.71MB / 231.14MB 1.1s #13 sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d1832dd2e0f2f8cf9e33 114.29MB / 231.14MB 1.2s #13 sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d1832dd2e0f2f8cf9e33 [131](https://github.com/docker-mailserver/docker-mailserver/actions/runs/3945342546/jobs/6752151872#step:7:135).76MB / 231.14MB 1.4s #13 sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d1832dd2e0f2f8cf9e33 152.70MB / 231.14MB 1.5s #13 sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d1832dd2e0f2f8cf9e33 170.92MB / 231.14MB 1.7s #13 sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d1832dd2e0f2f8cf9e33 188.74MB / 231.14MB 1.8s #13 sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d1832dd2e0f2f8cf9e33 214.96MB / 231.14MB 2.0s #13 sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d1832dd2e0f2f8cf9e33 229.64MB / 231.14MB 2.1s #13 sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d1832dd2e0f2f8cf9e33 231.14MB / 231.14MB 3.0s done #13 extracting sha256:cff7aab4088b8607efe477bdd4a9e0185bc8ed8339e171980897550e15390466 2.3s done #13 extracting sha256:2d8c7c5ce39eaa53a3997[135](https://github.com/docker-mailserver/docker-mailserver/actions/runs/3945342546/jobs/6752151872#step:7:139)d3c16598f489f99655024514de17424d064b03e4 done #13 extracting sha256:24cb207b181c6103680fa838d6c19b2baa1869aee6afcd891ce46b7b174f7b26 done #13 DONE 3.3s #13 FROM docker.io/clamav/clamav:latest@sha256:314c46478306f1bbf3216e2a8ca4b3cb87ba5dd1e14fe4d43f0e3d[137](https://github.com/docker-mailserver/docker-mailserver/actions/runs/3945342546/jobs/6752151872#step:7:141)12a4af1 #13 extracting sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d[183](https://github.com/docker-mailserver/docker-mailserver/actions/runs/3945342546/jobs/6752151872#step:7:187)2dd2e0f2f8cf9e33 #13 extracting sha256:06486e751a263a61f8582b3ef09ed4a80cdc2a3dbce6d1832dd2e0f2f8cf9e33 1.5s done #13 DONE 4.8s #14 [stage-base 3/49] RUN chmod +x /usr/local/bin/sedfile #14 CACHED #15 [stage-base 4/49] COPY target/scripts/build/* /build/ #15 CACHED #16 [stage-base 5/49] COPY target/scripts/helpers/log.sh /usr/local/bin/helpers/log.sh #16 CACHED #17 [stage-base 6/49] RUN /bin/bash /build/packages.sh #17 CACHED #18 [stage-base 2/49] COPY target/bin/sedfile /usr/local/bin/sedfile #18 CACHED #19 [stage-base 7/49] RUN </etc/cron.d/clamav-freshclam...) #19 CACHED ERROR: failed to solve: invalid user index: -1 ```
--- Related workflow logs for bug reporting:
Run docker/build-push-action@v3.3.0 ``` with: context: . platforms: linux/amd6[4](https://github.com/docker-mailserver/docker-mailserver/actions/runs/3945342546/jobs/6752151872#step:7:4) cache-from: type=local,src=/tmp/.buildx-cache cache-to: type=local,dest=/tmp/.buildx-cache-new,mode=max outputs: type=cacheonly load: false no-cache: false pull: false push: false github-token: *** ```
Docker info (20.10.22) ``` /usr/bin/docker version Client: Version: 20.10.22+azure-1 API version: 1.41 Go version: go1.18.9 Git commit: 3a2c30b63ab20acfcc3f3550ea756a0561655a77 Built: Thu Dec 15 15:37:38 UTC 2022 OS/Arch: linux/amd64 Context: default Experimental: true Server: Engine: Version: 20.10.22+azure-1 API version: 1.41 (minimum version 1.12) Go version: go1.18.9 Git commit: 42c8b314993e5eb3cc2776da0bbe41d5eb4b707b Built: Thu Dec 15 22:17:04 2022 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.5.16+azure-1 GitCommit: 2e3140a0e09d[28](https://github.com/docker-mailserver/docker-mailserver/actions/runs/3945342546/jobs/6752151872#step:7:30)8a9086474752b4478aa0964e7c runc: Version: 1.1.4 GitCommit: 5fd4c4d144137e991c4acebb2146ab1483a97925 docker-init: Version: 0.19.0 GitCommit: /usr/bin/docker info Client: Context: default Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc., 0.9.1+azure-2) compose: Docker Compose (Docker Inc., 2.15.1+azure-1) Server: Containers: 1 Running: 1 Paused: 0 Stopped: 0 Images: 20 Server Version: 20.10.22+azure-1 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: false userxattr: false Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 1 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc Default Runtime: runc Init Binary: docker-init containerd version: 2e3140a0e09d288a9086474752b4478aa0964e7c runc version: 5fd4c4d144137e991c4acebb2146ab1483a97925 init version: Security Options: apparmor seccomp Profile: default Kernel Version: 5.15.0-10[30](https://github.com/docker-mailserver/docker-mailserver/actions/runs/3945342546/jobs/6752151872#step:7:32)-azure Operating System: Ubuntu 20.04.5 LTS OSType: linux Architecture: x86_64 CPUs: 2 Total Memory: 6.781GiB Name: fv-az628-879 ID: J5HJ:OPXN:RYBU:[55](https://github.com/docker-mailserver/docker-mailserver/actions/runs/3945342546/jobs/6752151872#step:7:57)NP:DUWN:INRV:KGP3:GG3Y:CDQE:4K4D:LC24:LEZB Docker Root Dir: /var/lib/docker Debug Mode: false Username: githubactions Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false ```
Buildx version (0.9.1) ``` /usr/bin/docker buildx version github.com/docker/buildx 0.9.1+azure-2 ed00243a0ce2a0aee75311b06e32d33b44729689 /usr/bin/docker buildx build --cache-from type=local,src=/tmp/.buildx-cache --cache-to type=local,dest=/tmp/.buildx-cache-new,mode=max --iidfile /tmp/docker-build-push-r1dOWM/iidfile --output type=cacheonly --platform linux/amd64 --metadata-file /tmp/docker-build-push-r1dOWM/metadata-file . ```
--- ## Additional references - [`Dockerfile`](https://github.com/docker-mailserver/docker-mailserver/blob/fb82082cf17b9721c1749304e24fcef6630ca034/Dockerfile#L45) - [CI build workflow](https://github.com/docker-mailserver/docker-mailserver/blob/fb82082cf17b9721c1749304e24fcef6630ca034/.github/workflows/generic_build.yml) - [PR with single `Dockerfile` line change failing to build](https://github.com/docker-mailserver/docker-mailserver/pull/3011/files#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557R45)
polarathene commented 1 year ago

Minimal Reproduction

It was nothing to do with caching, my local build with BuildKit wasn't using buildx.

It appears to be specific to buildx with a builder using the docker-container driver. (Resolved: I used an outdated BuildKit from Docker Engine, buildx had the one with actual --link support)

Installed buildx locally (via package manager, v0.9.1), switched to docker-container driver and attempted a build but got a weird failure to troubleshoot:

Minimal Dockerfile without --chown ```Dockerfile # syntax=docker.io/docker/dockerfile:1 # Line above required for BUILDKIT features (RUN with HereDoc, and COPY with --link) FROM docker.io/debian:11-slim ARG DEBIAN_FRONTEND=noninteractive # Inline shell script to run: RUN <
Terminal output from failed build (Network error: "no such host") ```console $ docker buildx create --driver docker-container --name test-builder $ docker buildx build --builder test-builder . [+] Building 2.0s (6/6) FINISHED => [internal] load .dockerignore 0.3s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile 0.4s => => transferring dockerfile: 644B 0.0s => resolve image config for docker.io/docker/dockerfile:1 0.8s => CACHED docker-image://docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81 0.1s => => resolve docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81e24ef1a0dbc 0.1s => CANCELED [internal] load metadata for docker.io/clamav/clamav:latest 0.1s => ERROR [internal] load metadata for docker.io/library/debian:11-slim 0.0s ------ > [internal] load metadata for docker.io/library/debian:11-slim: ------ Dockerfile:4 -------------------- 2 | # Line above required for BUILDKIT features (RUN with HereDoc, and COPY with --link) 3 | 4 | >>> FROM docker.io/debian:11-slim 5 | ARG DEBIAN_FRONTEND=noninteractive 6 | -------------------- ERROR: failed to solve: failed to do request: Head "https://registry-1.docker.io/v2/library/debian/manifests/11-slim": dial tcp: lookup registry-1.docker.io on 192.168.1.1:53: no such host ```

Workaround was to create the driver with network in host mode: --driver-opt network=host, and now it builds (without the COPY --chown option):

Successful build without --chown ```console $ docker buildx create --driver docker-container --driver-opt network=host --name test-builder $ docker buildx build --builder test-builder . [+] Building 24.6s (12/12) FINISHED => [internal] load .dockerignore 0.6s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile 0.4s => => transferring dockerfile: 644B 0.0s => resolve image config for docker.io/docker/dockerfile:1 1.4s => CACHED docker-image://docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81 0.1s => => resolve docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81e24ef1a0dbc 0.1s => [internal] load metadata for docker.io/library/debian:11-slim 0.6s => [internal] load metadata for docker.io/clamav/clamav:latest 0.8s => FROM docker.io/clamav/clamav:latest@sha256:314c46478306f1bbf3216e2a8ca4b3cb87ba5dd1e14fe4d43f0e3d13712a4af1 0.1s => => resolve docker.io/clamav/clamav:latest@sha256:314c46478306f1bbf3216e2a8ca4b3cb87ba5dd1e14fe4d43f0e3d13712a4af 0.1s => CACHED [stage-0 1/3] FROM docker.io/library/debian:11-slim@sha256:98d3b4b0cee264301eb1354e0b549323af2d0633e1c433 0.2s => => resolve docker.io/library/debian:11-slim@sha256:98d3b4b0cee264301eb1354e0b549323af2d0633e1c43375d0b25c01826b6 0.1s => [stage-0 2/3] RUN < [stage-0 3/3] COPY --link --from=docker.io/clamav/clamav:latest /var/lib/clamav /var/lib/clamav 0.2s ```

Now Dockerfile has --chown=clamav added:

Minimal Dockerfile with COPY --chown ```Dockerfile # syntax=docker.io/docker/dockerfile:1 # Line above required for BUILDKIT features (RUN with HereDoc, and COPY with --link) FROM docker.io/debian:11-slim ARG DEBIAN_FRONTEND=noninteractive # Inline shell script to run: RUN <
Terminal output from failed build ("failed to solve: invalid user index: -1") ```console $ docker buildx create --driver docker-container --driver-opt network=host --name test-builder $ docker buildx build --builder test-builder . [+] Building 4.0s (9/9) FINISHED => [internal] load .dockerignore 0.1s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile 0.1s => => transferring dockerfile: 659B 0.0s => resolve image config for docker.io/docker/dockerfile:1 1.4s => CACHED docker-image://docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81 0.1s => => resolve docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81e24ef1a0dbc 0.1s => [internal] load metadata for docker.io/clamav/clamav:latest 0.5s => [internal] load metadata for docker.io/library/debian:11-slim 1.5s => CACHED FROM docker.io/clamav/clamav:latest@sha256:314c46478306f1bbf3216e2a8ca4b3cb87ba5dd1e14fe4d43f0e3d13712a4a 0.1s => => resolve docker.io/clamav/clamav:latest@sha256:314c46478306f1bbf3216e2a8ca4b3cb87ba5dd1e14fe4d43f0e3d13712a4af 0.1s => [stage-0 1/3] FROM docker.io/library/debian:11-slim@sha256:98d3b4b0cee264301eb1354e0b549323af2d0633e1c43375d0b25 0.2s => => resolve docker.io/library/debian:11-slim@sha256:98d3b4b0cee264301eb1354e0b549323af2d0633e1c43375d0b25c01826b6 0.2s => CACHED [stage-0 2/3] RUN <
polarathene commented 1 year ago

Observations:

The linked official docs for those options don't seem to describe any reason that shouldn't work (and it does work without docker-container driver). It's unclear why it works for the root user, but unable to reference anything else in /etc/passwd and /etc/group files in the container?

polarathene commented 1 year ago 
$ docker buildx create \
  --driver docker-container \
  --driver-opt network=host \
  --name test-builder

$ docker buildx build \
  --builder test-builder \
  --cache-from type=local,src=/tmp/cache-clamav \
  --cache-to type=local,dest=/tmp/cache-clamav,mode=max \
  --output type=cacheonly \
  .
Terminal output of build ```console # When already run previously, cache used: [+] Building 4.3s (21/21) FINISHED => [internal] load .dockerignore 0.1s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile 0.1s => => transferring dockerfile: 1.59kB 0.0s => resolve image config for docker.io/docker/dockerfile:1 0.8s => CACHED docker-image://docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81 0.1s => => resolve docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81e24ef1a0dbc 0.1s => [internal] load metadata for docker.io/clamav/clamav:latest 0.3s => [internal] load metadata for docker.io/library/debian:11-slim 0.7s => importing cache manifest from local:13317055486547944541 0.0s => FROM docker.io/clamav/clamav:latest@sha256:314c46478306f1bbf3216e2a8ca4b3cb87ba5dd1e14fe4d43f0e3d13712a4af1 0.1s => => resolve docker.io/clamav/clamav:latest@sha256:314c46478306f1bbf3216e2a8ca4b3cb87ba5dd1e14fe4d43f0e3d13712a4af 0.1s => [stage-clamav 1/4] FROM docker.io/library/debian:11-slim@sha256:98d3b4b0cee264301eb1354e0b549323af2d0633e1c43375 0.2s => => resolve docker.io/library/debian:11-slim@sha256:98d3b4b0cee264301eb1354e0b549323af2d0633e1c43375d0b25c01826b6 0.2s => CACHED [stage-base 2/2] RUN < CACHED [stage-clamav 4/4] RUN chown -R clamav:clamav /var/lib/clamav 0.0s => exporting content cache 1.3s => => preparing build cache for export 1.3s => => writing layer sha256:21bc083bb21a2ed3379881cdd9511756af15e1de0c42b4d833dbdc3a3e542c78 0.0s => => writing layer sha256:466b086beee78754cb7c1c0da94e767e728b44d8db77b80d844422f79202bc58 0.0s => => writing layer sha256:659b17fc892887fede20243c06f7ffbef10e4a1979ccfa433bf6e21bc63e477e 0.0s => => writing layer sha256:67b54556d93fb637dc1dd3005de8badec1ecc69badfc35c36f281223e549f3be 0.0s => => writing layer sha256:8740c948ffd4c816ea7ca963f99ca52f4788baa23f228da9581a9ea2edd3fcd7 0.0s => => writing layer sha256:d1c9d67ad5a967324c451e16e3664e804f2922fd651928fc6876eebb6122da62 0.0s => => writing config sha256:dcbd2a4efa9b69f46b282359b16ccb346fb7e7698808ca603a73ad99910e27b3 0.0s => => writing manifest sha256:f16b074b904d2ec5eef3c08af6e7eb44ec14058a2a8724866378112c8b689ed5 0.0s => CACHED [stage-clamav 2/4] COPY --link --from=stage-base /etc/passwd /etc/group /etc/ 0.0s => CACHED [stage-clamav 3/4] COPY --link --from=docker.io/clamav/clamav:latest /var/lib/clamav /var/lib/clamav 0.0s => CACHED [stage-final 1/1] COPY --link --from=stage-clamav /var/lib/clamav /var/lib/clamav 0.0s ```

Dockerfile:

# syntax=docker.io/docker/dockerfile:1
# Line above required for BUILDKIT features (RUN with HereDoc, and COPY with --link)

# Installs the clamav packages, which updates /etc/passwd and /etc/group with clamav user:
FROM docker.io/debian:11-slim as stage-base  
ARG DEBIAN_FRONTEND=noninteractive

# Inline shell script to run:
RUN <<EOF
  apt-get -y update 
  apt-get -y upgrade

  apt-get -y --no-install-recommends install clamav clamav-daemon ca-certificates

  rm -rf /var/lib/apt/lists/*
EOF

# 230MB x2 in size added from /var/lib/clamav
FROM docker.io/debian:11-slim as stage-clamav
# Reference user + group lookups from stage-base files instead:
COPY --link --from=stage-base /etc/passwd /etc/group /etc/

# Copy over the latest DB updates from the official ClamAV image.
# Better than running `freshclam` (which would require 500MB+ extra memory during an image build)
COPY --link --from=docker.io/clamav/clamav:latest /var/lib/clamav /var/lib/clamav
# NOTE: buildx with the docker-container driver does not support mixing COPY with `--link` and `--chown`,
# Unless using numeric UID / GID values. Handling this on a separate layer thus doubles the cache size:
RUN chown -R clamav:clamav /var/lib/clamav

# The next stage can then COPY the clamav data, so that the built image doesn't incur the chown size penalty:
# NOTE: Build cache will increase by approx 460MB from the separate stage, 
# Link here still incurs 230MB into build cache for this stage:
FROM stage-base as stage-final
COPY --link --from=stage-clamav /var/lib/clamav /var/lib/clamav
polarathene commented 1 year ago

Closing as incompatibility is expected (could do with a better error output and documentation due to the backwards-compatibility situation silently ignoring --link)

Possible workarounds

Intermediate stage

# syntax=docker.io/docker/dockerfile:1
# Line above required for BUILDKIT features (RUN with HereDoc, and COPY with --link)

# Installs the clamav packages, which updates /etc/passwd and /etc/group with clamav user:
FROM docker.io/debian:11-slim as stage-base  
ARG DEBIAN_FRONTEND=noninteractive

# Inline shell script to run:
RUN <<EOF
  apt-get -y update 
  apt-get -y upgrade

  apt-get -y --no-install-recommends install clamav clamav-daemon ca-certificates

  rm -rf /var/lib/apt/lists/*
EOF

# Copy over the latest DB updates from the official ClamAV image.
# Better than running `freshclam` (which would require 500MB+ extra memory during an image build)
FROM docker.io/debian:11-slim as stage-clamav
# Reference user + group lookups from stage-base files instead:
COPY --link --from=stage-base /etc/passwd /etc/group /etc/
COPY --chown=clamav --from=docker.io/clamav/clamav:latest /var/lib/clamav /var/lib/clamav

# NOTE: Build cache will increase by approx 230MB from the separate stage
# `--link` below still adds 230MB into the build cache for this stage (thus doubling build cache size needed):
FROM stage-base as stage-final
COPY --link --from=stage-clamav /var/lib/clamav /var/lib/clamav

--chown with numeric ID (add user in advance)

# syntax=docker.io/docker/dockerfile:1
# Line above required for BUILDKIT features (RUN with HereDoc, and COPY with --link)

FROM docker.io/debian:11-slim as stage-base  

RUN <<EOF
  DEBIAN_FRONTEND=noninteractive
  # Add the clamav user before installing the package, use an explicit UID to provide to COPY --chown
  adduser --quiet --system --group --disabled-password --home /var/lib/clamav --no-create-home --uid 200 clamav

  apt-get -y update 
  apt-get -y upgrade

  apt-get -y --no-install-recommends install clamav clamav-daemon ca-certificates

  rm -rf /var/lib/apt/lists/*
EOF

# Copy over the latest DB updates from the official ClamAV image.
# Better than running `freshclam` (which would require 500MB+ extra memory during an image build)
COPY --link --chown=200 --from=docker.io/clamav/clamav:latest /var/lib/clamav /var/lib/clamav