tonistiigi / binfmt

Cross-platform emulator collection distributed with Docker images.
MIT License
971 stars 70 forks source link

Request: update `latest` #165

Open AkihiroSuda opened 8 months ago

AkihiroSuda commented 8 months ago

The latest tag hasn't been updated since Aug 12, 2022 https://hub.docker.com/layers/tonistiigi/binfmt/latest/images/sha256-66ac1b854f9ce567503783aeee464ff1569076bf3613d122a5174890cbea34f9?context=explore

AkihiroSuda commented 8 months ago

@tonistiigi PTAL 🙏

crazy-max commented 8 months ago

@AkihiroSuda This is related to https://github.com/tonistiigi/binfmt/pull/120#issuecomment-1710586120.

Looking at this comment https://github.com/tonistiigi/binfmt/pull/120#discussion_r1237536156, args[] is not tampered so this is smth to fix in direct exec patches to not prepend the workdir: https://github.com/tonistiigi/binfmt/pull/120/commits/5da9dbfcc9b9b415194211a7ad3b267e68461659. So yeah direct without -P but this is not an issue with QEMU_PRESERVE_ARGV0, just changes needed in execve patches https://github.com/tonistiigi/binfmt/tree/master/patches/buildkit-direct-execve-v8.0 Probably https://github.com/tonistiigi/binfmt/blob/master/patches/buildkit-direct-execve-v8.0/0003-linux-user-path-in-execve-should-be-relative-to-work.patch

When fixed I think we can update latest.

norrisjeremy commented 7 months ago

What's the status on this? It doesn't inspire confidence that the official setup-qemu-action Github action is pointing at a Docker image that hasn't been updated in over 2 years.

tonistiigi commented 7 months ago

We have versioned releases for the updated versions (eg. 8.1.5) . Latest tag has not been updated because there were some backwards incompatible changes as mentioned above that need to be fixed (additionally there is also some breakage in new qemu version for tini https://github.com/tonistiigi/binfmt/blob/master/patches/subreaper-prctl/0001-linux-user-pass-SUBREAPER-to-prctl.patch ). If you don't care about these upstream issues you can just use the versioned release directly.

We still want to update to the latest tag, but only if we can be certain that it doesn't cause many issues to existing users.

norrisjeremy commented 7 months ago

Hi @tonistiigi,

It's not especially clear which tags are appropriate to use with the setup-qemu-action Github action. Is there any documentation that helps to provide clarity?

Thanks, Jeremy

tonistiigi commented 7 months ago

@crazy-max Maybe we can add an example in the docs for using the 8.1.5 tag.

norrisjeremy commented 7 months ago

Is there any guidance yet as to which 8.1.5 tag is appropriate to use with the setup-qemu-action Github action?

tonistiigi commented 7 months ago

You can use either qemu-v8.1.5 or desktop-v8.1.5. The difference is that latter has some extra patches to maintain better backwards compatibility https://github.com/tonistiigi/binfmt/blob/master/docker-bake.hcl#L83

norrisjeremy commented 7 months ago

You can use either qemu-v8.1.5 or desktop-v8.1.5. The difference is that latter has some extra patches to maintain better backwards compatibility https://github.com/tonistiigi/binfmt/blob/master/docker-bake.hcl#L83

FYI, neither of these appear to be usable with Github Actions, as they result in segfaults:

63.80 Processing triggers for libc-bin (2.35-0ubuntu3.6) ...
64.04 Segmentation fault (core dumped)
64.18 Segmentation fault (core dumped)
64.18 dpkg: error processing package libc-bin (--configure):
64.18  installed libc-bin package post-installation script subprocess returned error exit status 139
64.19 Errors were encountered while processing:
64.19  libc-bin
64.27 E: Sub-process /usr/bin/dpkg returned an error code (1)

This occurred simply executing apt-get dist-upgrade -y inside of an ubuntu:22.04 container.

tonistiigi commented 6 months ago

@norrisjeremy Under GH codespaces seemed to work fine

$ docker run -it --rm --privileged tonistiigi/binfmt:qemu-v8.1.5 --install arm64
$ docker buildx build --platform linux/arm64 .
[+] Building 5.0s (7/7) FINISHED                                                                                                       docker:default
 => [internal] load .dockerignore                                                                                                                0.3s
 => => transferring context: 2B                                                                                                                  0.0s
 => [internal] load build definition from Dockerfile                                                                                             0.3s
 => => transferring dockerfile: 83B                                                                                                              0.0s
 => [internal] load metadata for docker.io/library/ubuntu:22.04                                                                                  1.1s
 => [auth] library/ubuntu:pull token for registry-1.docker.io                                                                                    0.0s
 => [1/2] FROM docker.io/library/ubuntu:22.04@sha256:a6d2b38300ce017add71440577d5b0a90460d0e57fd7aec21dd0d1b0761bbfb2                            1.9s
 => => resolve docker.io/library/ubuntu:22.04@sha256:a6d2b38300ce017add71440577d5b0a90460d0e57fd7aec21dd0d1b0761bbfb2                            0.1s
 => => sha256:a6d2b38300ce017add71440577d5b0a90460d0e57fd7aec21dd0d1b0761bbfb2 1.13kB / 1.13kB                                                   0.0s
 => => sha256:462e829de9164b6c066246cddc265a936071744f689f0ea73daa92b4f9feb47e 424B / 424B                                                       0.0s
 => => sha256:7423357ed609f13ba90313f43454dc3026afb26476e14cb8b1dbb3eadb8a192c 2.31kB / 2.31kB                                                   0.0s
 => => sha256:d5a2ad729c09fbfdb259ae764f92188efc67a615ffde9bb34a46298d1edf3cd6 27.36MB / 27.36MB                                                 0.7s
 => => extracting sha256:d5a2ad729c09fbfdb259ae764f92188efc67a615ffde9bb34a46298d1edf3cd6                                                        0.8s
 => [2/2] RUN apt-get dist-upgrade -y                                                                                                            1.0s
 => exporting to image                                                                                                                           0.5s 
Nuru commented 3 months ago

@tonistiigi I started having issues similar to @norrisjeremy this past week, still using the default latest tag:

#6 [linux/arm64 2/9] RUN apt-get update     && apt-get install     -y -u --allow-downgrades     --no-install-recommends     kubectl-1.29     && apt-get clean     && rm -rf /var/lib/apt/lists/*
#6 1.306 Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
#6 1.613 Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
#6 1.620 Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
#6 1.726 Reading package lists...
#6 6.549 E: Method https has died unexpectedly!
#6 6.549 E: Sub-process https received a segmentation fault.

Note that this is a multi-platform build: linux/amd64,linux/arm64

Only the cross-platform RUN step fails. If building on a linux/amd64 host, the linux/amd64 step succeeds and the linux/arm64 step fails. If building on a linux/arm64 host, the linux/arm64 step succeeds and the linux/amd64 step fails.

This seems to have been triggered by something in the GitHub Action Runner v2.381.0 release https://github.com/actions/actions-runner-controller/pull/3684 which did not explicitly update anything, but pulled in the current versions of unpinned dependencies. Specifically, I am using what was built by this Dockerfile.

Meroje commented 2 months ago

This seems to have been triggered by something in the GitHub Action Runner v2.381.0 release actions/actions-runner-controller#3684 which did not explicitly update anything, but pulled in the current versions of unpinned dependencies. Specifically, I am using what was built by this Dockerfile.

I'm still getting curl segfaults when reverting to an earlier build of 2.317, doesn't look like it's the root cause to me.

update: updating the K8S nodes from AL2 to AL2023 fixed it, along with matching the base image with the runner, eg. building an ubuntu20 on an ubuntu20 runner, or 22 on 22.