buildx fails: exporting to client stops "half-way"

[andrewufrank]

I have problems exporting to client in a buildx operation. The command is: docker buildx build --tag andrewufrank/xfce:latest --platform linux/arm/v7,linux/arm64,linux/amd64 --push . it stops in the middle of the export to client operation at an arbitrary amount of transfer. in repeated invocations, the error occurs sooner or later, eventually timing out.

I have tried docker buildx build --tag andrewufrank/xfce:latest --platform linux/arm/v7,linux/arm64,linux/amd64 --output=type=local,dest=/home/frank/installation . which seems to have the same issue, but leaves the images in the CACHE and the upload for the command with push starts immediately and fails at arbitrary amount of transfer, meaning a smaller or larger portion of the three images is transferred (one run gives 192 174 and 171 MB, the next stops at 169,113, and 203 MB).

log

frank@oporto:~/dockerfile-x11docker-xfce$ docker buildx build --tag andrewufrank/xfce:latest --platform linux/arm/v7,linux/arm64,linux/amd64 --output=type=local,dest=/home/frank .
[+] Building 396.2s (11/12)                                                                   
[+] Building 397.4s (11/12)                                                                   
[+] Building 399.2s (11/12)                                                                   
 => [internal] load build definition from Dockerfile                                     0.1s
 => => transferring dockerfile: 2.51kB                                                   0.0s
 => [internal] load .dockerignore                                                        0.1s
 => => transferring context: 2B                                                          0.0s
 => [linux/amd64 internal] load metadata for docker.io/library/debian:buster             2.2s
 => [linux/arm/v7 internal] load metadata for docker.io/library/debian:buster            2.1s
 => [linux/arm64 internal] load metadata for docker.io/library/debian:buster             2.1s
 => CACHED [linux/arm64 1/2] FROM docker.io/library/debian:buster@sha256:1e74c92df24063  0.0s
 => => resolve docker.io/library/debian:buster@sha256:1e74c92df240634a39d050a5e23fb18f4  0.0s
 => [linux/arm64 2/2] RUN apt-get update && apt-mark hold iptables &&     env DEBIAN_  302.8s
 => CACHED [linux/arm/v7 1/2] FROM docker.io/library/debian:buster@sha256:1e74c92df2406  0.0s
 => => resolve docker.io/library/debian:buster@sha256:1e74c92df240634a39d050a5e23fb18f4  0.0s
 => [linux/arm/v7 2/2] RUN apt-get update && apt-mark hold iptables &&     env DEBIAN  275.5s
 => [linux/amd64 1/2] FROM docker.io/library/debian:buster@sha256:1e74c92df240634a39d05  0.0s
 => => resolve docker.io/library/debian:buster@sha256:1e74c92df240634a39d050a5e23fb18f4  0.0s
 => CACHED [linux/amd64 2/2] RUN apt-get update && apt-mark hold iptables &&     env DE  0.0s
 => exporting to client                                                                396.8s
 => => copying files linux/arm/v7 147.06MB                                             396.8s
 => => copying files linux/arm64 157.28MB                                              396.8s
 => => copying files linux/amd64 139.08MB     

I can add the Dockerfile, which i think does not contain anything unusual:

FROM debian:buster

RUN apt-get update && apt-mark hold iptables && \
    env DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
      dbus-x11 \
      psmisc \
      xdg-utils \
      x11-xserver-utils \
      x11-utils && \
    env DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
      xfce4 && \
    env DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
      gtk3-engines-xfce \
      libgtk-3-bin \
      libpulse0 \
      mousepad \
      xfce4-notifyd \
      xfce4-taskmanager \
      xfce4-terminal && \
    env DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
      xfce4-battery-plugin \
      xfce4-clipman-plugin \
      xfce4-cpufreq-plugin \
      xfce4-cpugraph-plugin \
      xfce4-diskperf-plugin \
      xfce4-datetime-plugin \
      xfce4-fsguard-plugin \
      xfce4-genmon-plugin \
      xfce4-indicator-plugin \
      xfce4-netload-plugin \
      xfce4-notes-plugin \
      xfce4-places-plugin \
      xfce4-sensors-plugin \
      xfce4-smartbookmark-plugin \
      xfce4-systemload-plugin \
      xfce4-timer-plugin \
      xfce4-verve-plugin \
      xfce4-weather-plugin \
      xfce4-whiskermenu-plugin && \
    env DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
      libxv1 \
      mesa-utils \
      mesa-utils-extra && \
    sed -i 's%<property name="ThemeName" type="string" value="Xfce"/>%<property name="ThemeName" type="string" value="Raleigh"/>%' /etc/xdg/xfce4/xfconf/xfce-perchannel-xml/xsettings.xml

CMD ["startxfce4"]

and my version:

Client: Docker Engine - Community
 Version:           19.03.12
 API version:       1.40
 Go version:        go1.13.10
 Git commit:        48a66213fe
 Built:             Mon Jun 22 15:45:50 2020
 OS/Arch:           linux/amd64
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          19.03.12
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.10
  Git commit:       48a66213fe
  Built:            Mon Jun 22 15:44:21 2020
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

[tonistiigi]

Although this should still work (haven't tested your Dockerfile) the fact that you are running such a command hints to me that something is wrong. The local exporter is for exporting the raw files to a directory and is usually used together with multi-stage to export out build artifacts like binaries. You are exporting whole debian rootfs with a bunch of extra software on top of it. Not only does it seem useless to have such directories of files, because the owner of all the files will be the local user this debain would not even boot if reconverted to another format. If you want to get an image offline that you can move to another machine use oci or docker exporter. If you really want to extract raw filesystem data use tar exporter that can preserve the original ownership values.

[andrewufrank]

thank you for clarification - indeed it was an error to use local exporter and I did obviously not undrerstand what it could be used for reasonable. Not likely useful for what I wanted to achieve! Thank you for clarification.

I was fighting with #327 and got there i received the perfect advice (from you)

Please test with moby/buildkit#1636 docker buildx create --driver-opt image=moby/buildkit:master

which solved all my problems!