search

rust-lang / wg-cargo-std-aware

Repo for working on "std aware cargo"
138 stars 8 forks source link

Cross compilation (of libc) on MacOS fails #66

Open raphaelcohn opened 3 years ago

raphaelcohn commented 3 years ago

I'm cross compiling for a linux musl target on MacOS, using the latest version of the homebrew musl-cross package. This works fine for normal work I do. However, when attempting to do a standard build, the libc rust crate fails with error: could not find native static libraryc, perhaps an -L flag is missing?. Rightly so, as it has been linked added as a dependency to rustc command line (long trace below).

This is because libc is odd; it doesn't emit use cargo:rustc-link-search in it's build script, build.rs. A patch to do this back in 2015 was rejected because it's 'fixed upstream' (https://github.com/rust-lang/libc/pull/17) in rust itself (https://github.com/rust-lang/rust/commit/52862e4cda1ff72e52dfaea21cf82ae93725b3be). I have to say, I didn't follow this reasoning.

This is a place where one would like to inject a custom flag - I know where and how to search for the libc - but I can only do so by modifying RUSTFLAGS, which affects everything else being built. Of course, I could write a rustc wrapper script, do a bit of argument sense checking, and then call rustc, but that's... work. (As an aside, when doing controlled, reproducible builds, I do this a lot, unsetting PATH and then providing shim wrappers for tools until I build up a profile of what's needed to do a reproducible build, especially if build tools call date and the like).

This seems related to #14 and other issues.

   Compiling alloc v0.0.0 (/Users/raph/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc)
     Running `CARGO=/Users/raph/.rustup/toolchains/nightly-x86_64-apple-darwin/bin/cargo CARGO_CRATE_NAME=alloc CARGO_MANIFEST_DIR=/Users/raph/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc CARGO_PKG_AUTHORS='The Rust Project Developers' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=alloc CARGO_PKG_REPOSITORY='' CARGO_PKG_VERSION=0.0.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=0 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' DYLD_FALLBACK_LIBRARY_PATH='/Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/target/release/deps:/Users/raph/.rustup/toolchains/nightly-x86_64-apple-darwin/lib:/Users/raph/.rustup/toolchains/nightly-x86_64-apple-darwin/lib:/Users/raph/lib:/usr/local/lib:/usr/lib' RUSTC_BOOTSTRAP=1 rustc --crate-name alloc --edition=2018 /Users/raph/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=d600ad6f2ddde8a8 -C extra-filename=-d600ad6f2ddde8a8 --out-dir /Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/target/x86_64-unknown-linux-musl-tiger-lake/release/deps --target /Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/x86_64-unknown-linux-musl-tiger-lake.json -C linker=x86_64-linux-musl-cc -Z force-unstable-if-unmarked -L dependency=/Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/target/x86_64-unknown-linux-musl-tiger-lake/release/deps -L dependency=/Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/target/release/deps --extern compiler_builtins=/Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/target/x86_64-unknown-linux-musl-tiger-lake/release/deps/libcompiler_builtins-e59aa6ff3d802cf9.rmeta --extern core=/Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/target/x86_64-unknown-linux-musl-tiger-lake/release/deps/libcore-0e79ae0d5145d2a2.rmeta --cap-lints warn -Z verbose=yes -Z print-link-args=yes -L native=c`
error: could not compile `libc`

Caused by:
  process didn't exit successfully: `CARGO=/Users/raph/.rustup/toolchains/nightly-x86_64-apple-darwin/bin/cargo CARGO_CRATE_NAME=libc CARGO_MANIFEST_DIR=/Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/vendor/libc-0.2.79 CARGO_PKG_AUTHORS='The Rust Project Developers' CARGO_PKG_DESCRIPTION='Raw FFI bindings to platform libraries like libc.
  ' CARGO_PKG_HOMEPAGE='https://github.com/rust-lang/libc' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=libc CARGO_PKG_REPOSITORY='https://github.com/rust-lang/libc' CARGO_PKG_VERSION=0.2.79 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=2 CARGO_PKG_VERSION_PATCH=79 CARGO_PKG_VERSION_PRE='' DYLD_FALLBACK_LIBRARY_PATH='/Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/target/release/deps:/Users/raph/.rustup/toolchains/nightly-x86_64-apple-darwin/lib:/Users/raph/.rustup/toolchains/nightly-x86_64-apple-darwin/lib:/Users/raph/lib:/usr/local/lib:/usr/lib' OUT_DIR=/Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/target/x86_64-unknown-linux-musl-tiger-lake/release/build/libc-19f238fd5225b055/out RUSTC_BOOTSTRAP=1 rustc --crate-name libc /Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/vendor/libc-0.2.79/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg 'feature="align"' --cfg 'feature="rustc-dep-of-std"' --cfg 'feature="rustc-std-workspace-core"' -C metadata=a26b5536f1af61b7 -C extra-filename=-a26b5536f1af61b7 --out-dir /Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/target/x86_64-unknown-linux-musl-tiger-lake/release/deps --target /Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/x86_64-unknown-linux-musl-tiger-lake.json -C linker=x86_64-linux-musl-cc -Z force-unstable-if-unmarked -L dependency=/Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/target/x86_64-unknown-linux-musl-tiger-lake/release/deps -L dependency=/Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/target/release/deps --extern rustc_std_workspace_core=/Volumes/Source/lemonrock/linux-support/rust-custom-target.workspace/target/x86_64-unknown-linux-musl-tiger-lake/release/deps/librustc_std_workspace_core-a9fcb53369c5d212.rmeta --cap-lints warn -Z verbose=yes -Z print-link-args=yes -L native=c --cfg freebsd11 --cfg libc_priv_mod_use --cfg libc_union --cfg libc_const_size_of --cfg libc_align --cfg libc_core_cvoid --cfg libc_packedN --cfg libc_cfg_target_vendor --cfg libc_thread_local` (exit code: 1)
12101111 commented 3 years ago 
error: could not find native static library `c`, perhaps an -L flag is missing?

error: aborting due to previous error

error: could not compile `libc`

Caused by:
  process didn't exit successfully: `CARGO=/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/bin/cargo CARGO_CRATE_NAME=libc CARGO_MANIFEST_DIR=/home/han/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.88 CARGO_PKG_AUTHORS='The Rust Project Developers' CARGO_PKG_DESCRIPTION='Raw FFI bindings to platform libraries like libc.
  ' CARGO_PKG_HOMEPAGE='https://github.com/rust-lang/libc' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=libc CARGO_PKG_REPOSITORY='https://github.com/rust-lang/libc' CARGO_PKG_VERSION=0.2.88 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=2 CARGO_PKG_VERSION_PATCH=88 CARGO_PKG_VERSION_PRE='' LD_LIBRARY_PATH='/usr/obj/rust/debug/deps:/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib:/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib' OUT_DIR=/usr/obj/rust/x86_64-unknown-linux-musl/debug/build/libc-4cf2a7bffddd2e5d/out RUSTC_BOOTSTRAP=1 rustc --crate-name libc /home/han/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.88/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 --cfg 'feature="align"' --cfg 'feature="rustc-dep-of-std"' --cfg 'feature="rustc-std-workspace-core"' -C metadata=d244570cd2e0e60e -C extra-filename=-d244570cd2e0e60e --out-dir /usr/obj/rust/x86_64-unknown-linux-musl/debug/deps --target x86_64-unknown-linux-musl -Z force-unstable-if-unmarked -L dependency=/usr/obj/rust/x86_64-unknown-linux-musl/debug/deps -L dependency=/usr/obj/rust/debug/deps --extern rustc_std_workspace_core=/usr/obj/rust/x86_64-unknown-linux-musl/debug/deps/librustc_std_workspace_core-597cd0340bdb6a9d.rmeta --cap-lints warn -Clinker=ld.lld -Ctarget-feature=-crt-static --cfg freebsd11 --cfg libc_priv_mod_use --cfg libc_union --cfg libc_const_size_of --cfg libc_align --cfg libc_core_cvoid --cfg libc_packedN --cfg libc_cfg_target_vendor --cfg libc_thread_local` (exit status: 1)

It also failed on a musl system

12101111 commented 3 years ago

A workaround for this is:

export RUSTFLAGS="-Lnative=/usr/lib"

(replace /usr/lib with the path to libc.a)

And a fix would be like https://github.com/rust-lang/libc/pull/1327 plus some adjustment in rustbuild like https://github.com/rust-lang/rust/pull/85600 to copy libc.a into self-contained directory.

Follpvosten commented 3 years ago

This also happens for me on x86_64-unknown-linux-gnu; you can clearly see that it tries to compile core and std from the wrong toolchain.