Closed jdrouet closed 3 years ago
Same here. I ran your example repo according to the instructions and it fails for me with the exact same error. I figured out that it seems to be a problem with 32bit systems on a 64bit host whilst doing research. Something about larger inodes so I guess cargo's first file system operation just blows up.
I have the same issue. Attaching my Dockerfile
. I tried to build it for armv7
on my amd64
machine. Anyone knows of any workaround?
FROM rust:1 as build
WORKDIR /usr/src/starship
ARG VERSION=0.47.0
RUN git clone https://github.com/starship/starship.git . \
&& git checkout tags/v$VERSION
RUN cargo build --release
RUN strip target/release/starship
FROM scratch AS binaries
COPY --from=build /usr/src/starship/target/$TARGETPLATFORM/release/starship /
You can take a look here. I do a fetch using the build platform arch and then copy it in the target platform.
I've the exact same error but in my builder image : I've tried rust:buster, debian:latest, python:3, ubuntu... to no avail...
Any idea I'm blocked ? (Or is there a catch in @jdrouet 's solution that I've missed? )
I cannot use a JFS volume as shown in the following link because the error happens at build time : https://itsze.ro/blog/2020/11/29/cross-compile-for-raspberry-pi-with-docker.html
@nicobo in my previous message I give this link in which I use multi stage builds to fix the problem ;)
@jdrouet yes I'm using multistage like in your Dockerfile but the problem happens in the build stage so I don't even reach the step where I could COPY --from
...
I'm not using --platform=$BUILDPLATFORM
(I let docker select the right arch), does that make a difference ?
if you don't use --platform=$BUILDPLATFORM
it will use the target platform. By specifying --platform=$BUILDPLATFORM
you force it to build with your computer's platform and you won't have the bug from qemu.
@jdrouet Thanks for the example Dockerfile - worked like a charm!
Not really related to buildx
, but does anyone know how to pass the BUILDPLATFORM
to the Dockerfile when building the Dockerfile with docker-compose
? Up to now, I've always used docker-compose build
when I wanted to quickly build a docker image on my machine and only used buildx
for building images for multiple architectures.
But with the --platform=$BUILDPLATFORM
workaround from above, docker-compose build
always fails with: failed to parse platform : "" is an invalid component of "": platform specifier component must match "^[A-Za-z0-9_-]+$": invalid argument
.
I already tried: docker-compose build --build-arg "BUILDPLATFORM=linux/amd64"
but unfortunately that doesn't work either - same error.
It's not a big deal for me, as I can use buildx
also for my local builds, but I was just wondering if anyone has an idea?
edit: I figured it out. COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose build
works for me (just make sure that your docker-compose
version is not too old)
if you don't use
--platform=$BUILDPLATFORM
it will use the target platform. By specifying--platform=$BUILDPLATFORM
you force it to build with your computer's platform and you won't have the bug from qemu.
Right, I managed to build it thanks to your example ! I'm not a Rust developer : I'm trying to compile someone else's Rust project ; so I didn't figured out that it was a Rust-specific solution.
As I understand it now (if useful to anyone) :
cargo fetch
, this step needs to be executed inside the working container (cargo vendor
will do it)vendor
, .cargo
) into the target imagecargo build ... --offline
), using the previously cached filesI have "vendor" files that look like architecture-specific (e.g. winapi-i686-pc-windows-gnu
) so I will see if it works at runtime when I can test a container.
I wonder then if Rust would simply allow us to fully cross-compile for all targets, then just copy the generated files into the final image?
My process is a little bit different since I first clone the code of an existing project from github, so I don't need to do cargo init
. The final Dockerfile looks like this and it builds fine : https://gist.github.com/nicobo/06530acc4fd82497878d45a93348ae3c
Thanks @jdrouet !
@nicobo That's basically what I am doing now. Always building the Rust dependencies when building my docker image took too long, so precompiled them with cross and injected the right shared object into my docker build.
see https://github.com/bbernhard/signal-cli-rest-api/tree/master/ext/libraries/zkgroup
and https://github.com/bbernhard/signal-cli-rest-api/blob/master/Dockerfile#L18
@bbernhard awesome this is libzkgroup I needed to build ! I noticed your image on Docker Hub but I've missed the cross-compiled binaries... from now on I can directly use yours !
@bbernhard awesome this is libzkgroup I needed to build ! I noticed your image on Docker Hub but I've missed the cross-compiled binaries... from now on I can directly use yours !
what a coincidence :grinning: - glad it's helpful to you! :)
Hello.
I am now reaching another level since this error arises when building a Python program, which depends on cryptography, a library which in turn builds with Rust... See : pyca/cryptography#5771
Let me explain the Python way : on some architectures (here linux/arm/v7) there is no "Python Wheel" package for this dependency so the pip install
command builds it from source... using Rust... triggering again a Value too large... error !
The problem is that I don't think I can alter the way this dependency builds when triggered by a pip install (the cargo build command is out of hand), and anyway I'm getting tired to implement patches making my Dockerfiles hard to maintain...
Any news ? Is there any link to an original issue on qemu or anything ?
I've disabled rust in the cryptography build for now but this trick will be removed from their next release...
You can test with QEMU from https://github.com/tonistiigi/binfmt that is made for running containers:
$ docker run -it --rm tonistiigi/binfmt:latest --version
binfmt/8703596 qemu/v6.0.0 go/1.16.5
Let us know if it still happens
@crazy-max it looks like the issue is still present, I’m building a docker image for multiple platforms, and it fails when I include linux/arm/v7 as can be seen here: https://github.com/texthtml/midi-synthetizer-autoconnect/runs/3399891145?check_suite_focus=true
I’m using the docker/setup-qemu-action@v1 which is using the Qemu binaries from https://github.com/tonistiigi/binfmt
I've tried using Java and JLink and it occurred the same error: log.txt
Caused by: java.nio.file.FileSystemException: /opt/java/openjdk/jmods: Value too large for defined data type
[linux/arm/v7]
qemu-v6.1.0-20
https://github.com/MarcelCoding/luna/blob/0b567aa6e1e7889bfe8ad3295dbf9dac1a855cf3/docker/Dockerfile.github-actions#L17-L23 https://github.com/MarcelCoding/luna/blob/0b567aa6e1e7889bfe8ad3295dbf9dac1a855cf3/.github/workflows/ci.yaml#L214-L227
@MarcelCoding were you able to fix or workaround the issue ?
unfortunately not, I just removed armhf support. (arm64 also wan't working, for that I removed the stip-debug option)
I was able to use armv6 and aarch64 just fine, its just arm7 causing this weird issue. Looks like manually building is the way to go for now
Somewhere I read that this is an Issue with qemu emulating an 32bit system on an 64bit host.
I see, thanks for the info :D
Get Outlook for Androidhttps://aka.ms/AAb9ysg प्राप्त करें
From: Marcel @.> Sent: Sunday, November 21, 2021 10:10:28 PM To: docker/buildx @.> Cc: Debayan Sutradhar @.>; Comment @.> Subject: Re: [docker/buildx] Value too large for defined data type (#395)
Somewhere I read that this is an Issue with qemu emulating an 32bit system on an 64bit host.
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/docker/buildx/issues/395#issuecomment-974851616, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGERF5LMF6LEGDKVOHYPZ23UNEOHXANCNFSM4RQIMLBA.
@MarcelCoding, you are right. If you follow ehuss' comment you end up at the QEMU bugtracker which goes into more detail for those interested.
Unfortunately the fix seems very far away, if there even will be a fix. Over in the cargo repo we got around the issue by mounting a tmpfs on the ~/.cargo
folder, and this is appears to not trigger the bug while compiling cryptography
anymore. Perhaps this is something that would work here as well.
I'm trying to build a Rust project for amd64, armv7 and armv8 locally, but when I do, I get the following error:
I put a repository here for you to reproduce it.
I don't really know if it's buildx related or buildkit related (most probably buildkit). The image builds well for armv7 on an raspberry pi, but not from my linux machine.