Cross-compiling from aarch64 to musl arm64

emk / rust-musl-builder

Docker images for compiling static Rust binaries using musl-libc and musl-gcc, with static versions of useful C libraries. Supports openssl and diesel crates.

Apache License 2.0

1.54k stars 193 forks source link

Cross-compiling from aarch64 to musl arm64 #117

Closed TilBlechschmidt closed 2 years ago

TilBlechschmidt commented 3 years ago

Okay, so now that we've got "build as root" support it is time for the next adventure 😉

TL;DR Apple has shiny new MacBooks with an aarch64 CPU. While the Docker Tech Preview is able to run amd64 images, qemu is really unhappy about it (especially when compiling). Thus, seemingly half the rust crates I use simply don't compile.

The most obvious solution would be to provide an aarch64 version of this image which is able to cross-compile to amd64. I've done some preliminary research and experimentation and came to the following milestones being required:

[ ] Replace the musl-cc compiler pulled from apt (musl-dev package presumably) with one from musl.cc for the appropriate target architecture (namely x86_64-linux-musl cross).
[ ] Tell cargo to use the correct linker using the config
[ ] Figure out how to compile OpenSSL and other libs

In my little POC I got the first two working but cheated by removing OpenSSL from my crate. If possible, I could use some assistance there.

TilBlechschmidt commented 3 years ago

In theory, it shouldn't even hurt changing the compiler in the arm64 image too. As we normally pull it in anyways. Since the base image used does have an aarch64 version and all apt packages that are pulled in are available too, this should be mostly plain sailing in terms of integration (again, apart from the compilation of other libs).

EDIT: Although the cargo extensions pulled in at the beginning (mdbook, deny, about) would probably have to be moved to a cargo install / conditionalised.

TilBlechschmidt commented 3 years ago

Update: You should probably run the test-image yourself to confirm my results. I'm getting some weird flacky behaviour and don't have any more time to investigate this evening.

emk commented 2 years ago

I strongly recommend switching to cross to support cross-compilation. It's supported by an official Rust team, and it works fine for projects which don't require libpq or OpenSSL. Much of the Rust ecosystem now works fine with rustls, and it's a very nice dev experience.

I'm going to close this issue as out-of-scope, but if you're interested in pursuing it, I'm potentially willing to move this project to a multi-maintainer organization.

emk commented 2 years ago

(And of course, many thanks for the attempt, and the notes on your progress so far!)

TilBlechschmidt commented 2 years ago

I'll take a look into the current state of rustls (IIRC I had issues with it previously as it couldn't handle certs for IP ranges but that might have changed). Thank you for the project and have a merry Christmas 🎄🎁

emk commented 2 years ago

Issues with cert support almost always come down to the webpki crate.

I was talking to the webpki maintainer the other day about certs without a proper alt name, and I got the impression that PRs would be welcome.