Open fenollp opened 5 months ago
Looks like you are trying to write full container rootfs (or maybe even docker state directory, unclear what the working dir in your example is). This likely contains many system files and files that require specific permissions, and you don't have permissions to create such files on the client side.
There is almost never a valid use case to export full container rootfs (1.4GB) like this. If you want to export the files as one squashed bundle instead of exporting image you should use -o type=tar
that exports the same files as local output would do but in a tar container so it doesn't need to replicate the individual files on client side.
Ok so -o type=tar
works. I no longer have the above issue.
0 0s ~ λ DOCKER_HOST=ssh://oomphy docker build --build-arg ARGs='https://www.youtube.com/watch?v=zg4Q0xtcFlg' --output=type=tar,dest=$HOME/try.tar https://github.com/fenollp/dockerhost-tools--yt-dlp.git#cfd938b53f072da5fcdeaa9526435ae4577a25c0
[+] Building 0.0s (0/0) docker:default
[+] Building 63.9s (15/15) FINISHED docker:default
=> CACHED [internal] load git source https://github.com/fenollp/dockerhost-tools--yt-dlp.git#cfd938b53f072da5fcdeaa9526435ae4577a25c0 0.0s
=> resolve image config for docker.io/docker/dockerfile:1@sha256:39b85bbfa7536a5feceb7372a0817649ecb2724562a38360f4d6a7782a409b14 0.0s
=> CACHED docker-image://docker.io/docker/dockerfile:1@sha256:39b85bbfa7536a5feceb7372a0817649ecb2724562a38360f4d6a7782a409b14 0.0s
=> [internal] load metadata for docker.io/library/alpine@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1 0.0s
=> [alpine 1/1] FROM docker.io/library/alpine@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1 0.0s
=> CACHED [tool 1/3] RUN --mount=type=cache,target=/var/cache/apk ln -vs /var/cache/apk /etc/apk/cache && set -ux && apk add ffmpeg gcc git 0.0s
=> CACHED [atomicparsley 1/2] WORKDIR /app 0.0s
=> CACHED [atomicparsley 2/2] RUN set -ux && wget https://github.com/wez/atomicparsley/releases/download/20221229.172126.d813aa6/AtomicParsleyAlpine.zip && unzip AtomicParsleyA 0.0s
=> CACHED [tool 2/3] COPY --from=atomicparsley /app/AtomicParsley /usr/local/bin/ 0.0s
=> CACHED [tool 3/3] RUN set -ux && echo --force-ipv4 >>/etc/yt-dlp.conf && echo "--output '%(title).200s-%(id)s.%(ext)s'" >>/etc/yt-dlp.conf && echo --audio-multistreams >>/e 0.0s
=> CACHED [product 1/3] WORKDIR /app 0.0s
=> CACHED [product 2/3] RUN --mount=type=cache,target=/root/.cache/yt-dlp set -ux && cmd="yt-dlp --cache-dir /root/.cache/yt-dlp --newline" && case "https://www.youtube.com 0.0s
=> CACHED [product 3/3] RUN set -ux && if [ "${DO_NOT_REENCODE:-}" != '1' ]; then while read -r vid; do ffmpeg -i "$vid" -force_key_frames 'expr:gte(t,n_forced*3)' 0.0s
=> CACHED [stage-4 1/1] COPY --from=product /app/* / 0.0s
=> exporting to client tarball 63.3s
=> => sending tarball 63.2s
0 70s ~ λ tar tvf try.tar
-rw-r--r-- 0/0 1443814029 2024-01-29 17:05 La Commune 1871 (1ère partie) - Peter Watkins-zg4Q0xtcFlg.mp4
However I'm not sure I understand the other elements of your response:
$HOME/.local/share/docker/overlay/..
) isn't something I'm trying to write to (however my -o
does point to $HOME
). Could that be a problem?rootfs
: the stage I copy things from is FROM scratch
type=tar
as it looses progress info (just says "exporting to client tarball") and I'd like final files to overwrite my local files (or to only copy non-locally available files)Please clear up these points :)
Note: using --output=/tmp/
I get this (only after downloading all the data):
=> CACHED [stage-4 1/1] COPY --from=product /app/* / 0.0s
=> ERROR exporting to client directory 72.8s
=> => copying files 1.44GB 72.7s
------
> exporting to client directory:
------
ERROR: failed to solve: error from receiver: open /tmp/snap-private-tmp: permission denied
IMO the issue I describe is a bug (it used to work!) BUT I'm nozing that my install may be the culprit.
Contributing guidelines
I've found a bug and checked that ...
Description
Running plenty of jobs with
build --output=$DIR ..
whileDOCKER_HOST=ssh://..
is set on a rootless client, executing the jobs on a (non-rootless) buildx remote, I often getexport
errors:Expected behaviour
I expect the final export to properly download a non-zero sized file back to my buildx client.
Actual behaviour
The whole download happens (progress is shown but
export
fails right at the end, due to wrong permissions under$HOME/.local/share/docker/overlay2
Buildx version
github.com/docker/buildx v0.12.1 30feaa1
Docker info
Builders list
Configuration
Build logs
Additional info
The last folder in the path it errors on is
world-unreadable
: