search

rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
96.56k stars 12.47k forks source link

Cross-compilation from LInux to x86_64-apple-darwin fails due to bad "cc" command line params #112501

Open John-Nagle opened 1 year ago

John-Nagle commented 1 year ago

Rust compilation on Ubuntu 22.04.2 LTS with target x86_64-apple-darwin fails with a build error:

cc: error: unrecognized command-line option '-arch' cargo generated a cc command that can't work.

cc is (Ubuntu 11.3.0-1ubuntu1-22.04.1) 11.3.0, installed with Ubuntu 22.04.2 LTS.

Reproduce by:

rustup target add x86_64-apple-darwin
cargo new hello_world
cd hello_world
cargo build

Expected a successful cross-compile.

Meta

rustc --version --verbose:

rustc 1.69.0 (84c898d65 2023-04-16)
binary: rustc
commit-hash: 84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc
commit-date: 2023-04-16
host: x86_64-unknown-linux-gnu
release: 1.69.0
LLVM version: 15.0.7

Discussion

It's clear why this doesn't work. Cargo/rustc is invoking the platform's linker in a cross-compile, and the default linker is not capable of that cross-platform action. So this is a tool dependency problem. "rustup" installed support for that target with no errors, and if dependency checks needed to be made, they were not made at that time.

Note that the hard work is done. The compile is succesful. It's only the linking that does not work.

It can be argued that this particular cross-compilation pair is not supported. However, both the compiling platform and the target platform are Rust Tier 1 supported targets. "Tier 1 targets can be thought of as 'guaranteed to work'." There are no footnotes to the target table indicating that this source->target combination does not work.

Policy on target support says that "Tier 2 targets should, if at all possible, support cross-compiling. Tier 2 targets should not require using the target as the host for builds, even if the target supports host tools." That's for Tier 2. This is a Tier 1 target. For Tier 1 with Host Tools, the policy notes "Providing host tools does not exempt a target from requirements to support cross-compilation if at all possible."

So, if this cross-compile is technically possible, it is a bug that it does not work.

It is technically possible, since at least two unsupported workarounds exist.

Workarounds

There are at least two unsupported workarounds for this, using different tool chains.

  1. Osxcross This is a set of scripts that collects tools capable of this build. This approach may require the use of proprietary Apple, Inc. libraries, which is inconsistent with Rust policy.
  2. Zig Using components of the Zig toolchain appears to work. This approach does not seem to require any proprietary Apple, Inc. code.

Industry trends

Cross-platform support is becoming more widespread. Microsoft recently announced that their Windows OS is now fully supported on some Apple hardware. Apple is now offering a game porting toolkit for converting Windows games to Apple targets. The "walled garden" model seems to be weakening, at least for desktops. Zig, a language which competes in Rust's space, already supports this cross-compile.

So this cross-compile should be fully supported. Thank you.

Jules-Bertholet commented 1 year ago

@rustbot label A-cross O-linux O-macos

jyn514 commented 1 year ago

The compiler appears to be unconditionally using the linker-flavor of the target instead of the flavor of the host: https://github.com/rust-lang/rust/blob/b80e0b7f53461a827fa007910e86560eec3a1180/compiler/rustc_target/src/spec/apple_base.rs#L101-L104

jyn514 commented 1 year ago

ah, this won't work anyway because the macOS SDK needs to be installed: https://github.com/tpoechtrager/osxcross#how-does-it-work

John-Nagle commented 1 year ago

The Zig compiler people somehow got past that.

workingjubilee commented 1 year ago

Sure, let's just ask @andrewrk if this was before or after the custom linker or if it was in fact overcome as opposed to merely avoided.

workingjubilee commented 1 year ago

In any case, the platform support policy has only been enforced strictly for new targets. For "grandfathered" targets like x86_64-apple-darwin, there are de facto deviations and failures.

This is much more obvious for x86_64-pc-windows-msvc, which is Windows 7, but it is not in fact tested to the standard of a tier 1 platform, i.e. "tests pass on every commit". It's not much of a test if it doesn't test the actual version, and CI tests on Windows 10 as those are the CI images we can actually obtain. "Creatively" interpreting the policy is considered more acceptable than suddenly pulling the plug on everyone who still wants to ship Rust binaries to Windows 7 and Windows 8. It is explicitly footnoted, but that does not change a policy deviation being a deviation. Likewise there is no formal maintainer team for most of the current batch of tier 1 targets.

Anyways, it seems that the Zig cross-compilation story is not magic or bulletproof:

Which I am not going to cast aspersions on! It merely shows an immediate problem for us: Rust is much more pedantic about UTF-8 than Zig is and it is... very beneficial to our binary size if we can use the system libraries. Merely having more struggles linking libiconv seems like a bad start. "You can cross-compile from Linux or Windows to macOS but it doubles your starting binary size compared to a native build" sounds plausible as something we might want to offer, but also would be a Big Oof if that is the tradeoff we would have to make.

Presumably this issue can still be solved rather than "fixed", but it seems effortlessly cross-compiling a programming language with a different set of features than Zig has to macOS is not an already-solved problem with no tradeoffs.

andrewrk commented 1 year ago

For iconv, you should be able to solve it the same way as libSystem, which is to ship the tbd files with the Rust toolchain. These can be given to the linker to produce dynamic dependencies on the target system's iconv shared libraries. Note that Zig does not provide Unicode APIs in its standard library, nor does the language depend on Unicode, so iconv is simply not a dependency of most Zig projects.

For frameworks, our plan is to rely on third party packages that provide them rather than shipping them directly with zig. The Rust equivalent I believe would be a Cargo crate for each framework.

The headers/libs things looks like a silly bug that I'm not sure how it hasn't been fixed yet, and the unwinding thing looks like a nightmare level linker bug that we'll unfortunately have to address eventually.

workingjubilee commented 1 year ago

Wonderful! Thank you for the info, @andrewrk, that will make improving the situation here much easier. The "framework crate" answer sounds pretty close to how things work-out in practice with the ecosystem's convention of Rust {lib}-sys crates.

Also, I have been debriefed more on the background issues for libiconv:

So it looks like we don't need a dependency on libiconv: Rust provides its own UTF-8 handling, so we aren't even benefiting from libiconv for binary size! It's purely an accident! Kind of annoying.

tindzk commented 1 year ago

Besides Zig, you could try rust-lld which is bundled with the Rust toolchain. I am using these commands in GitHub Actions (ubuntu-latest) to build for macOS:

curl -L https://github.com/roblabla/MacOSX-SDKs/releases/download/13.3/MacOSX13.3.sdk.tar.xz | tar xJ
export SDKROOT=$(pwd)/MacOSX13.3.sdk/
export PATH=$PATH:~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/
export CARGO_TARGET_X86_64_APPLE_DARWIN_LINKER=rust-lld
rustup target add x86_64-apple-darwin
cargo build --release --target x86_64-apple-darwin
csaben commented 9 months ago

Besides Zig, you could try rust-lld which is bundled with the Rust toolchain. I am using these commands in GitHub Actions (ubuntu-latest) to build for macOS:

curl -L https://github.com/roblabla/MacOSX-SDKs/releases/download/13.3/MacOSX13.3.sdk.tar.xz | tar xJ
export SDKROOT=$(pwd)/MacOSX13.3.sdk/
export PATH=$PATH:~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/
export CARGO_TARGET_X86_64_APPLE_DARWIN_LINKER=rust-lld
rustup target add x86_64-apple-darwin
cargo build --release --target x86_64-apple-darwin

you are a legend.

John-Nagle commented 9 months ago

I'll have to try that.

It turns out that my ui-mock program, which exercises the Egui/Rend3/WPGU/Winit stack, ran properly on MacOS when built by someone who had a Mac. On Apple silicon, even. No Mac-related mods required. So I'll have to try a cross-compile.

Portability is working better than expected. I build the Windows version on Linux and release it that way.

drHyperion451 commented 8 months ago

Besides Zig, you could try rust-lld which is bundled with the Rust toolchain. I am using these commands in GitHub Actions (ubuntu-latest) to build for macOS:

curl -L https://github.com/roblabla/MacOSX-SDKs/releases/download/13.3/MacOSX13.3.sdk.tar.xz | tar xJ
export SDKROOT=$(pwd)/MacOSX13.3.sdk/
export PATH=$PATH:~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/
export CARGO_TARGET_X86_64_APPLE_DARWIN_LINKER=rust-lld
rustup target add x86_64-apple-darwin
cargo build --release --target x86_64-apple-darwin

I just used export CARGO_TARGET_X86_64_APPLE_DARWIN_LINKER=rust-lld on my mac and it worked perfectly

charro commented 7 months ago

Besides Zig, you could try rust-lld which is bundled with the Rust toolchain. I am using these commands in GitHub Actions (ubuntu-latest) to build for macOS:

curl -L https://github.com/roblabla/MacOSX-SDKs/releases/download/13.3/MacOSX13.3.sdk.tar.xz | tar xJ
export SDKROOT=$(pwd)/MacOSX13.3.sdk/
export PATH=$PATH:~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/
export CARGO_TARGET_X86_64_APPLE_DARWIN_LINKER=rust-lld
rustup target add x86_64-apple-darwin
cargo build --release --target x86_64-apple-darwin

This is working great, thanks for sharing! My 2 cents: If you want to use directly rust Docker image rust:latest in your CI instead of ubuntu, I'm doing so adding what's missing for the cross-build:

apt-get update -yqq
apt-get install -yqq --no-install-recommends build-essential
apt-get install -yqq mingw-w64   # For Windows build
apt-get install -yqq clang gcc g++ zlib1g-dev libmpc-dev libmpfr-dev libgmp-dev cmake libxml2-dev libssl-dev zip unzip
piaoger commented 3 months ago

Above method really works. But in my case which may use some c/c++ deps, there will have below errors:

  ...
  cargo:warning=cc: error: unrecognized command line option ‘-arch’
  cargo:warning=cc: error: x86_64: No such file or directory
  cargo:warning=cc: error: unrecognized debug output level ‘full’
  cargo:warning=cc: error: unrecognized command line option ‘-arch’
 ...
error: unable to find framework 'CoreFoundation'. searched paths:  none

Just uses SDKROOT + zigbuild !!

FROM

cargo build --release --target x86_64-apple-darwin

TO

cargo zigbuild --release --target x86_64-apple-darwin
John-Nagle commented 3 months ago

Oh, nice. The Zig people are putting in effort on cross-platform. I'll have to try that.

Clownsw commented 2 months ago

That is a great act.