Rust cannot use universal LLVM on macOS

[MarcusCalhoun-Lopez]

I install Rust using MacPorts on macOS. Rust was recently updated from 1.24.1 to 1.25.0. For version 1.24.1, the Rust compiler did not seem to care if LLVM was built as a universal binary. Starting with version 1.25.0, I get the error

error: failed to add native library /opt/local/libexec/llvm-6.0/lib/libLLVMX86Disassembler.a: File too small to be an archive

Rust was compiled with --llvm-root=/opt/local/libexec/llvm-6.0. If I reinstall LLVM so it is not universal, Rust compiles correctly. There is a MacPorts bug report.

[raphaelcohn]

@MarcusCalhoun-Lopez @awakecoding I get this too, but with libraries built by cmake using the rust cmake crate in a Cargo build.rs script. I can probably introduce a workaround step, but this is a compiler bug that is really unfortunate - and misreports the cause of the error.

[awakecoding]

@raphaelcohn you might be in luck then, have you tried setting the CMAKE_OSX_ARCHITECTURES to a single architecture in your build.rs? If it doesn't fix it, check that the project CMakeLists.txt does not overwrite the CMAKE_OSX_ARCHITECTURES value. It doesn't change the fact that the problem with the Rust linker should definitely be fixed, but in your case you may be able to work around it for now.

[raphaelcohn]

@awakecoding Thanks for the hint - I'll try that out. For now, I'm parsing the cargo TARGET environment variable and putting in some hacks to run lipo -thin (lots of hacks, actually, as lipo uses arm64 and rust uses aarch64, that sort of thing). And it's rather brittle. It'll certainly break in a cross-compilation...

[awakecoding]

@raphaelcohn you can use CARGO_CFG_TARGET_OS and CARGO_CFG_TARGET_ARCH to make a pretty accurate mapping between names used by cargo and names used by CMake. It's glue layer code, but I'm pretty sure if you do the mapping correctly just for macOS and iOS to specify a single parameter to CMAKE_OSX_ARCHITECTURES it should work, saving you the pain of calling lipo. In my case, since the dependency is prebuilt, I guess my only choice at this point would be to add checks for macOS + iOS, call lipo -thin to extract just the right library and copy it to a temporary location, using this library instead of the original. Such a pain :/

[raphaelcohn]

@awakecoding Thanks for the extra tip - that's awesome! I must have missed those. For now, my problem's actually compounded a little that the third party library I'm wrapping (IntelSEAPI) is basically a mess, with a very broken build / install system that makes way to many assumptions, most of them wrong...

[raphaelcohn]

@awakecoding Definitely a pain - and it just makes a reproducible process that much harder. Not least the mental distraction from whatever problem you were actually trying to solve with rust...

[awakecoding]

@raphaelcohn tell me about it - broken build systems? totally unheard of ;) At Devolutions I ended up developing a structured collection of python modules to build all of our dependencies or projects correctly for Windows, macOS, Linux, Android and iOS, and we have about 35 of those. We use cmake extensively to simplify importing/exporting complex native dependencies. However, many third-party cmake projects don't do modern cmake exporting correctly, if they even do it at all, so I patched a bunch of them on a branch that we use to build. This year our objective would be to retire this in-house collection of scripts in favor of conan, which pretty much has everything we could ever wish for. The cool thing with conan is that it doesn't force a specific build system, you just write a python recipe to build the project and tell it what build artifacts it produces (it has helpers for cmake). From there, conan can magically provide the glue necessary to import those artifacts back into whatever build system you want to use, may it be cmake, visual studio, xcode or even cargo in the future.

Now one thing I would really like to see is a generic way to provide build artifact information to a Rust "-sys" crate. I originally wanted to extract all of my native dependency information from cmake in a top-level build.rs and pass it down to modified "-sys" crates but it turns out it cannot be done. A crate you depend on can pass environment variables back to crates depending on it, but not the other way around. I don't mind this rule, but if it would be possible to pass something like a file instead that would fit the format of what the build.rs outputs, it might be a suitable option.

I like Rust and Cargo, but there are definitely things that need improving to simplify the integration of prebuilt third-party dependencies in Rust crates, especially if the dependency is shared with existing C/C++ code and Rust.

[raphaelcohn]

@awakecoding Seems we think on similar lines... I've made two build systems over the years, one worse, one better. These days I think of how to achieve maximum reproducibility and make sure everything - even the toolchain - is reproducible, and how to bootstrap from minimal dependencies. Hence my Libertine Linux distro effort.

Having put a little thought into this weekend, I wonder if it would be possible to develop standard cross-compilation toolchains (ie ones that have no host dependencies) for the common targets Rust supports, and then ship cmake toolchain files, GNU autoconf m4 macro cruft, etc so -sys files could use them - and provide a standard way to 'override' these toolchains for certain builds. A bit of a nightmare to get going with, but certainly do-able for moderately sane stuff (eg musl), not least because one's shipping what C has always lacked...

[AregevDev]

Having the same issue as the issue referenced above, trying to build a library with cmake-rs but it fails telling me it is too small to be an archive. The output library seems fine though Job log: https://api.travis-ci.com/v3/job/196405004/log.txt

[ryandesign]

Really frustrating that this bug still hasn't been fixed.

[awakecoding]

With Apple making the switch to ARM for macOS, we'll probably see a lot more universal binaries come up for macOS, not just iOS.

[cormacrelf]

Correct, another repro on an M1 machine, that's currently stopping me from building MaterializeInc/materialize.

git clone https://github.com/fede1024/rust-rdkafka.git && cd rust-rdkafka
git co -b repro d7c2baf3
cargo build --features=cmake-build

Produces the 'file too small' confusion:

error: failed to add native library /Users/.../src/rust-rdkafka/target/debug/build/rdkafka-sys-9611bbf5b4a9f22a/out/lib/librdkafka.a: file too small to be an archive

Where the file in question is a universal binary:

$ file target/debug/build/rdkafka-sys-9611bbf5b4a9f22a/out/lib/librdkafka.a
target/debug/build/rdkafka-sys-9611bbf5b4a9f22a/out/lib/librdkafka.a: Mach-O universal binary with 2 architectures: [x86_64:current ar archive random library] [arm64]
target/debug/build/rdkafka-sys-9611bbf5b4a9f22a/out/lib/librdkafka.a (for architecture x86_64): current ar archive random library
target/debug/build/rdkafka-sys-9611bbf5b4a9f22a/out/lib/librdkafka.a (for architecture arm64):  current ar archive random library

Info

$ rustc --version # installed via rustup
rustc 1.51.0-nightly (04caa632d 2021-01-30)

[cormacrelf]

Ok, being a linker problem it's mainly an issue for cargo, and that's being tracked already: https://github.com/rust-lang/cargo/issues/8875. Maybe this one stays open because of the way it make rustc fail to build, but #55235 can probably be closed.

[nachkar]

Any solution to build the rust library for both architecture arm64 and x86 using Apple LLVM compiler not rust's ? Bitcode enabled is affecting the release of the application due to the difference between the LLVM producer and reader while archiving on xCode.