Open leet4tari opened 1 month ago
to use a custom image with another platform you need to specify the platforms it can/should run with. See https://github.com/cross-rs/cross/blob/main/docs/config_file.md#targettargetimage
[target.aarch64-unknown-linux-gnu]
image.name = "alpine:edge"
image.toolchain = ["x86_64-unknown-linux-musl", "linux/arm64=aarch64-unknown-linux-musl"]
so in this case, you want
[target.aarch64-unknown-linux-gnu]
image.name = "my-image"
image.toolchain = ["aarch64-unknown-linux-gnu"]
Thanks for the reply @Emilgardis.
I had tried quite a few option and I might not have listed trying this:
[target.aarch64-unknown-linux-gnu]
image = "ubuntu:18.04"
pre-build = "./scripts/cross_compile_hack.sh"
[target.aarch64-unknown-linux-gnu.env]
passthrough = [
"CARGO_BUILD_TARGET",
"CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc",
"BINDGEN_EXTRA_CLANG_ARGS=--sysroot /usr/aarch64-linux-gnu/include/",
]
and my script does a uname -m
which is listing x86 and not aarch64 as I had expected. Basic idea was that on MacOS, could use cross-rs and docker to cross-compile Linux binaries, with minimal fuss and effect.
The problem I am seeing, is that cross-rs is not letting me choose the docker platform to run/build on, even if I try and overload with either of the two env OPTS
.
You need to tell cross that the image can run on aarch64 platform, we assume x86_64 if nothing is specified.
To choose the platform, you need to add it as a valid platform for that image
[target.aarch64-unknown-linux-gnu]
image.name = "ubuntu:18.04"
image.toolchain = ["aarch64-unknown-linux-gnu"]
There is however no way to tell cross "of all the possible platforms, please use x
", we should probably make that possible
Thanks again @Emilgardis , can we not use the env CROSS_CONTAINER_OPTS
to force docker to use a platform? That is something I saw suggested in an issue while looking through this.
What I was raising is that the docker run command, thou it does add the env to the run but at the end, does not replace what looks like the default of docker buildx build --platform linux/amd64
vs replacing the --platform
part if put into the env.
What my original expectation was that the docker would be running with local CPU arch, unless overridden or forced, might that be better user expectation?
To not have cross insert --platform
if it's already been specified with CROSS_CONTAINER_OPTS
you'd have to modify https://github.com/cross-rs/cross/blob/19be83481fd3e50ea103d800d72e0f8eddb1c90c/src/docker/image.rs#L437
I think we can do that, it's consistent with what we do for --load
on CROSS_BUILD_OPTS
. There might however be other issues with that due to the expectation in the code that we've determined the platform and are using it.
What my original expectation was that the docker would be running with local CPU arch, unless overridden or forced, might that be better user expectation?
That's an interesting insight, it might be worth doing. My only concern is how moby/buildx now attests image provenence with unknown/unknown
, which together with the issue presented in #1214 makes it kinda confusing.
That is the issue where I got the idea to try and force the docker platform, when I was able to debug that the arm64 was defaulting to using the x86_64 docker engine.
The idea I was raising and really hoping would be easy to fix or even test, would be to stop the defaulting to x86_64 or be able to use the --platform
override, but my rust is not good enough to come up with sort of patch. Happy to try and test anything, if you or anybody else has some suggestions,.
Interestingly, somebody might want to force building with qemu
or rosetta
that is not the default docker target, but that is a different problem, I would just like to be able to use the native CPU arch, but been able to target a different OS, ie osx -> linux.
As for the docker images having unknown/unknown
, I still don't understand this behaviour and have disable for now in my own projects.
It's unclear to me if you've been able to solve your problem, the way to immediately solve your issue of not being able to specify --platform=linux/arm64
is the config I shared earlier, does that work for now?
With that config it's even possible to specify precisely what platform to use
[target.aarch64-unknown-linux-gnu]
image.name = "ubuntu:18.04"
# syntax = <platform>=<toolchain>
image.toolchain = ["linux/arm64=aarch64-unknown-linux-gnu"]
I missed your suggestion on how to possible target the platform, let me test and report back. Thanks @Emilgardis for all the support.
Thanks @Emilgardis , I was able to come right. I can cross-compile our project on both x86_64 and aarch64 OSX computers to Linux x86_64 and aarch64.
I am not sure I would have tried your suggestion from the documentation, thou that might just be me trying to get something do in a very specific kind of way.
Still not sure that it is right, but it works.
The docker command line might still be a bug, as it does not only issue one set of args as overrides, just duplicates what you have put into the env.
Adding some info and possible see which way this goes
[target.aarch64-unknown-linux-gnu]
image = "ubuntu:18.04"
# Need to use native aarch64 on Apple silicon
#image.name = "ubuntu:18.04"
#image.toolchain = ["linux/arm64=aarch64-unknown-linux-gnu"]
pre-build = "./scripts/cross_compile_ubuntu_18-pre-build.sh"
[target.x86_64-unknown-linux-gnu]
image = "ubuntu:18.04"
pre-build = "./scripts/cross_compile_ubuntu_18-pre-build.sh"
maybe better for me to open another issue regarding how the Apple Silicon does not seem to use the native platform and need to use the commented out targeting using the toolchain.
Using the selection as is, looks like it works on Windows and Linux, but they both x86_64 systems, vs the Apple machine which is native aarch64
. I have to use the commented section instead to force docker image to use aarch64
image, else the build does not work.
The image.toolchain config is an array, you can specify multiple platforms/toolchains
It works like that by design, as I've described earlier
So, you would do
image.toolchain = ["linux/arm64=aarch64-unknown-linux-gnu", "linux/amd64=x86_64-unknown-linux-gnu"]
It's unclear to me if you've been able to solve your problem, the way to immediately solve your issue of not being able to specify
--platform=linux/arm64
is the config I shared earlier, does that work for now?With that config it's even possible to specify precisely what platform to use
[target.aarch64-unknown-linux-gnu] image.name = "ubuntu:18.04" # syntax = <platform>=<toolchain> image.toolchain = ["linux/arm64=aarch64-unknown-linux-gnu"]
When I try this, I get:
❯ cross --version
cross 0.2.5
Error:
0: failed to parse file `"bla/Cross.toml"` as TOML
1: invalid type: map, expected a string for key `target.aarch64-unknown-linux-gnu.image` at line 2 column 14
@KarstenB I should clarify, above only works on the main branch, image.toolchain
has not been released yet
so, you'll need to install cross as cargo install cross --git https://github.com/cross-rs/cross
Checklist
Describe your issue
I don't think that the
CROSS_CONTAINER_OPTS
env is replacing the--platform linux/amd64
command, at least not on Apple Silicon.What target(s) are you cross-compiling for?
aarch64-unknown-linux-gnu, x86_64-unknown-linux-gnu
Which operating system is the host (e.g computer cross is on) running?
What architecture is the host?
What container engine is cross using?
cross version
cross 0.2.5 (19be834 2024-05-17)
Example
Running
then trying run
You might notice
docker buildx build --platform linux/amd64
with--platform linux/arm64
much later in the command, which I think force running x86 images, even thou I would like to use an aarch64/arm64 image.Reason been, that I was looking to run a CPU native docker image as my base to cross-compile, I was expecting/hoping to use
aarch64-unknown-linux-gnu
image (Ubuntu 18.04), but kept running intorosetta
errors.Additional information / notes
No response