MacOS fat binaries support

[xTachyon]

I'm trying to compile a project under MacOS that sets -DCMAKE_OSX_ARCHITECTURES="arm64;x86_64" in order to create a fat binary. The code compiles, but in the linking step, it fails with the below message, plus a bunch of undefined symbols:

ld: warning: ignoring file /Users/.../therustlib.a, building for macOS-arm64 but attempting to link with file built for macOS-x86_64

I've tried messing with Rust_CARGO_TARGET, but every time the build fails on a mismatch or another. I've even tried putting both the targets in that variable, but it doesn't seem like it's something that the corrosion accepts.

I've tried detecting the arch that we're currently compiling to with stuff like CMAKE_SYSTEM_PROCESSOR, but this seems to only be ever set to x86_64.

[jschwe]

As far as I know cargo/rust doesn't support compiling fat binaries (unless this changed recently)

[xTachyon]

A possible solution I can see is compiling both versions separately, and then merging them with lipo. I think that's what the xcode generator is doing.

I've tried doing that manually for a test, and hardcoding the .a path in cmake, and it seems to have worked. Maybe corrosion can automate this?

[jschwe]

s compiling both versions separately, and then merging them with lipo.

What is the scope here? Does this work for all rust artifacts corrosion supports, i.e. for staticlibs AND for cdylibs AND for bin targets? Your example seems to imply you only tested with staticlibs.

In principle it would be possible for corrosion to support this, but i'd have to give some more thought on how much refactoring is required to allow this. Currently, the assumption of "one target triple" is pretty baked in.

If you just want a quick solution for your own project, then you could probably just configure two new sub-cmake projects from your main cmake project, and add a rule in the main cmake project to use lipo to merge the artifacts from the sub cmake project.

[xTachyon]

Initially I just tested with staticlibs, but just now I tested with a new hello world project with bin, staticlib and cdylib by building them twice and invoking lipo on them and it seems to be working on all the artifact types. sh for cdylib:

cargo b --target x86_64-apple-darwin &&
cargo b --target aarch64-apple-darwin &&
lipo -create target/x86_64-apple-darwin/debug/libhello_world.dylib target/aarch64-apple-darwin/debug/libhello_world.dylib -output libhello_world_fat.dylib

file output:

% file libhello_world_fat.dylib
libhello_world_fat.dylib: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit dynamically linked shared library x86_64] [arm64:Mach-O 64-bit dynamically linked shared library arm64]
libhello_world_fat.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64
libhello_world_fat.dylib (for architecture arm64):  Mach-O 64-bit dynamically linked shared library arm64

The syntax for lipo seems to be identical in all 3 cases, I just changed the input and output filenames and it seems to pick up what they are automagically. So yes, it could work with all already supported crate types.

About your quick solution, could you please elaborate more please? What I understood right now was that I should do 2x corrosion_import_crate, and then do the lipo step myself, which sounded like a good workaround, but importing the same crate twice doesn't seem to be supported because "a target with the same name already exists".

[jschwe]

Thanks for testing!

What I understood right now was that I should do 2x corrosion_import_crate

No that doesn't work, because corrosion currently can only import each target once.

What I had in mind was something like

add_custom_command(OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/arm64-subbuild/mylib.a"
     COMMAND ${CMAKE_COMMAND} -S "${CMAKE_CURRENT_SOURCE_DIR}/rust"
                 -B "${CMAKE_CURRENT_BINARY_DIR}/arm64-subbuild"
                 -DRust_CARGO_TARGET=....
     COMMAND ${CMAKE_COMMAND} --build "${CMAKE_CURRENT_BINARY_DIR}/arm64-subbuild"
)

This custom command would be added once for each target and assumes that "${CMAKE_CURRENT_SOURCE_DIR}/rust" contains a CMakeLists.txt (with its own project() ) which adds corrosion and calls corrosion_import_crate().

In your main CMake files you could then add another custom command which DEPENDs on the output files of the commands, and calls lipo on them.

[xTachyon]

Found out you can also pass --target multiple times to cargo, and it will know how to build multiple arches at a time. In parallel even!

[jschwe]

Nice find - do you happen to know starting with which Rust version this works?

Edit: Seems to be Rust 1.64

[xTachyon]

Looks like 1.64.

Edit: you found first while I was searching😋.