Fails to link verbatim libary in 1.82 on macOS

anatawa12 commented 3 weeks ago

Code

Full project to reproduce this bug is at https://github.com/anatawa12/rust-1.82-linker-regression

I tried to link .a file at $OUT_DIR/testlib.a with the following build.rs code.


fn main() {
    let out_dir = PathBuf::from(std::env::var_os("OUT_DIR").unwrap());
    std::fs::create_dir_all(&out_dir).unwrap();
    let object = out_dir.join("testlib.o");
    let archive = out_dir.join("testlib.a");

    // buiid code here

    println!(
        "cargo:rustc-link-search=native={path}",
        path = out_dir.display()
    );
    println!("cargo:rustc-link-lib=static:+verbatim=testlib.a");
}

I ran cargo +stable test, linker reported ld: library 'testlib.a' not found, but with rust 1.81.0 (cargo +1.81.0 test) no linker error was reported.

Rust 1.82.0 on linux works well AFAIK (not tested with this code above, but works with real-world case I found this regression)

full build log with verbose

``` $ cargo +stable test --verbose Compiling rust-1-82-linker-regression v0.1.0 (/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression) Running `/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/bin/rustc --crate-name build_script_build --edition=2021 build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=192 --crate-type bin --emit=dep-info,link -C embed-bitcode=no --check-cfg 'cfg(docsrs)' --check-cfg 'cfg(feature, values())' -C metadata=d7bb1aa767972a7a -C extra-filename=-d7bb1aa767972a7a --out-dir /Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/build/rust-1-82-linker-regression-d7bb1aa767972a7a -C incremental=/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/incremental -L dependency=/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/deps` Running `/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/build/rust-1-82-linker-regression-d7bb1aa767972a7a/build-script-build` Running `/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/bin/rustc --crate-name rust_1_82_linker_regression --edition=2021 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=192 --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 -C split-debuginfo=unpacked --check-cfg 'cfg(docsrs)' --check-cfg 'cfg(feature, values())' -C metadata=add4a5facfd2f7ab -C extra-filename=-add4a5facfd2f7ab --out-dir /Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/deps -C incremental=/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/incremental -L dependency=/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/deps -L native=/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/build/rust-1-82-linker-regression-a47462767d058db3/out -l 'static:+verbatim=testlib.a'` Running `/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/bin/rustc --crate-name rust_1_82_linker_regression --edition=2021 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=192 --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C split-debuginfo=unpacked --test --check-cfg 'cfg(docsrs)' --check-cfg 'cfg(feature, values())' -C metadata=963aac10ba26161b -C extra-filename=-963aac10ba26161b --out-dir /Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/deps -C incremental=/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/incremental -L dependency=/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/deps -L native=/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/build/rust-1-82-linker-regression-a47462767d058db3/out -l 'static:+verbatim=testlib.a'` error: linking with `cc` failed: exit status: 1 | = note: env -u IPHONEOS_DEPLOYMENT_TARGET -u TVOS_DEPLOYMENT_TARGET -u XROS_DEPLOYMENT_TARGET LC_ALL="C" PATH="/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/bin:/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/bin:/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/bin:/Users/anatawa12/.cargo/bin:/Users/anatawa12/.opam/default/bin:/opt/homebrew/opt/coreutils/libexec/gnubin:/opt/homebrew/opt/grep/libexec/gnubin:/Users/anatawa12/.pyenv/shims:/Users/anatawa12/.rbenv/bin:/Users/anatawa12/.rbenv/shims:/opt/local/bin:/opt/local/sbin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/X11/bin:/Library/Apple/usr/bin:/Applications/Wireshark.app/Contents/MacOS:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Users/anatawa12/.cargo/bin:/opt/n/bin:/Users/anatawa12/bin:/Users/anatawa12/.dotnet/tools:/Users/anatawa12/go/bin" VSLANG="1033" ZERO_AR_DATE="1" "cc" "-arch" "arm64" "/var/folders/kw/yhxddc7x2_b9f__07m789msm0000gn/T/rustcxMuggb/symbols.o" "/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/deps/rust_1_82_linker_regression-963aac10ba26161b.230w3ltx1tyin48jyz9gg62ve.rcgu.o" "/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/deps/rust_1_82_linker_regression-963aac10ba26161b.2kpsu0kkm1iu0u33iw506w5qe.rcgu.o" "/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/deps/rust_1_82_linker_regression-963aac10ba26161b.2rra8qibc8xluhhphpeac63l6.rcgu.o" "/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/deps/rust_1_82_linker_regression-963aac10ba26161b.63fa677y8o4kupyjrhw2e06nv.rcgu.o" "/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/deps/rust_1_82_linker_regression-963aac10ba26161b.dozsgtx5r5kuvil9ikjcppamh.rcgu.o" "/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/deps/rust_1_82_linker_regression-963aac10ba26161b.4mrr65f7zswfitye8l3bs5h4p.rcgu.o" "-ltestlib.a" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libtest-85a9da6da381f636.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libgetopts-78018899fd21502d.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libunicode_width-869d479eca56b68b.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_std-4410c8ee795fd46e.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd-0b4a354a5d882f18.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libpanic_unwind-00e89274fccf37d9.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libobject-16cb3fa9562569e5.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libmemchr-726032628236814d.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libaddr2line-3a2050392888e443.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libgimli-22481e5f0a7ce87c.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_demangle-99efda3e7d60c7d0.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd_detect-97279fd1ed4043e6.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libhashbrown-14aec4517faeb95b.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_alloc-ce8f05ca22908841.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libminiz_oxide-45050aa8ac83ac5c.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libadler-3a0b6babb2efec9c.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libunwind-41b79f23e35577db.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcfg_if-7831295d10c91204.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liblibc-f86e480d5dcc309d.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liballoc-a7504b44dda8a2a3.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_core-589a9d977c5a484f.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcore-a17e2a568e77fc15.rlib" "/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcompiler_builtins-791901bf82ad4f12.rlib" "-lSystem" "-lc" "-lm" "-L" "/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/build/rust-1-82-linker-regression-a47462767d058db3/out" "-o" "/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/deps/rust_1_82_linker_regression-963aac10ba26161b" "-Wl,-dead_strip" "-nodefaultlibs" = note: ld: library 'testlib.a' not found clang: error: linker command failed with exit code 1 (use -v to see invocation) error: could not compile `rust-1-82-linker-regression` (lib test) due to 1 previous error Caused by: process didn't exit successfully: `/Users/anatawa12/.rustup/toolchains/stable-aarch64-apple-darwin/bin/rustc --crate-name rust_1_82_linker_regression --edition=2021 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=192 --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C split-debuginfo=unpacked --test --check-cfg 'cfg(docsrs)' --check-cfg 'cfg(feature, values())' -C metadata=963aac10ba26161b -C extra-filename=-963aac10ba26161b --out-dir /Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/deps -C incremental=/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/incremental -L dependency=/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/deps -L native=/Users/anatawa12/RustroverProjects/rust-1-82-linker-regression/target/debug/build/rust-1-82-linker-regression-a47462767d058db3/out -l 'static:+verbatim=testlib.a'` (exit status: 1) ```

Version it worked on

It most recently worked on: Rust 1.80 or 1.81 beta, and 1.81 stable on github actions

Version with regression

rustc --version --verbose:

rustc 1.82.0 (f6e511eec 2024-10-15)
binary: rustc
commit-hash: f6e511eec7342f59a25f7c0534f1dbea00d01b14
commit-date: 2024-10-15
host: aarch64-apple-darwin
release: 1.82.0
LLVM version: 19.1.1

Backtrace

No Compiler Crashes

Backtrace

``` ```

anatawa12 commented 3 weeks ago

other crates using this lib crate will build successfully. it fails only for tests on the crate

ehuss commented 3 weeks ago

Looks like this was changed in #128400 to remove a backwards compatibility hack for tests. Maybe @petrochenkov could explain why it also affects verbatim?

anatawa12 commented 3 weeks ago

https://github.com/rust-lang/rust/blob/32b17d56eb02495f9865028e1f7271a3a48c0b9b/compiler/rustc_codegen_ssa/src/back/linker.rs#L587-L594

I found that rustc solves path for archive with find_native_static_library for osx-like platforms if whole-archive enabled but rustc does not if whole-archive is disabled. this might be related.

I think it's very confusing that works for library crates and binary crates using them, but not working for test (or possibly binary crates with build.rs) builds.

madsmtm commented 3 weeks ago

I don't think this is specific to cargo test? Replacing lib.rs with a main.rs, I get the same error, and I get the error regardless of Rust version.

I notice that rustc is passing -ltestlib.a to the macOS linker, but that's just wrong? ld64 only supports verbatim .o in that position - the docs even call this difference in ld64 out, but the code doesn't seem to implement it...

anatawa12 commented 3 weeks ago

I don't think this is specific to cargo test? Replacing lib.rs with a main.rs, I get the same error, and I get the error regardless of Rust version.

Sorry I did not test that.

rust-lang / rust