Regression in running tests with custom library search path

[Riateche]

Steps to reproduce:

1. Make a crate that links to a native library which is not in system path. The crate's build script adds the library's path using cargo:rustc-link-search=native=.
2. Make a test in the crate and run it using cargo test.

cargo 0.17 (shipped with rust 1.16.0) successfully runs the test because it adds the library path to LD_LIBRARY_PATH (on Linux).

But cargo 0.18 (shipped with rust 1.17.0) fails to run the test:

Running [path]/[test_name]-f25c1f4d7cd45ca4 --nocapture [path]/[test_name]-f25c1f4d7cd45ca4: error while loading shared libraries: libctrt1.so: cannot open shared object file: No such file or directory

It doesn't add the library path to LD_LIBRARY_PATH anymore. The binary is linked successfully because cargo probably still supplies the path to the linker, but it's not enough to run the binary.

[alexcrichton]

Cargo was tweaked to not modify LD_LIBRARY_PATH for shared objects found outside the build directory as those tend to be system-level concerns, but do you know whether that's the case here? Is the shared object here inside or outside the build directory?

[Riateche]

In my case, the library is outside the build directory. What was the reason for this tweak? Are there any downsides in the old behavior?

[alexcrichton]

This was changed in https://github.com/rust-lang/cargo/pull/3651 which has some linked discussion for where this can go awry.

[retep998]

Did the issue fixed by #3651 also occur on Windows and Linux or was it a Mac exclusive issue? If it's the latter, then why did we change the behavior on Windows and Linux to break existing setups?

Frankly, I find it odd that we still conflate directories needed at link time with directories needed at run time. If we created a new cargo: key specifically for directories that need to be added to PATH or equivalent, then I'd have no problem with deprecating cargo:rustc-link-search for that purpose, and we wouldn't need to be discussing whether to add shared objects outside the build directory to PATH.

[lolgesten]

I got bitten by this today.

$ echo build.rs
...
    pkg_config::probe_library("openh264").unwrap();
...
$ pkg-config --libs openh264
-L/usr/local/lib -lopenh264

$ cargo test     
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
     Running /usr/src/umar/target/debug/deps/video-3f0ed96ec783fc2e
/usr/src/umar/target/debug/deps/video-3f0ed96ec783fc2e: error while loading 
shared libraries: libopenh264.so.4: cannot open shared object file: No such file or 
directory
error: test failed, to rerun pass '--lib'

Changing the LD_LIBRARY_PATH before running did fix the problem.

[stale[bot]]

As there hasn't been any activity here in over 6 months I've marked this as stale and if no further activity happens for 7 days I will close it.

I'm a bot so this may be in error! If this issue should remain open, could someone (the author, a team member, or any interested party) please comment to that effect?

The team would be especially grateful if such a comment included details such as:

• Is this still relevant?
• If so, what is blocking it?
• Is it known what could be done to help move this forward?

Thank you for contributing!

(The cargo team is currently evaluating the use of Stale bot, and using #6035 as the tracking issue to gather feedback.)

If you're reading this comment from the distant future, fear not if this was closed automatically. If you believe it's still an issue please leave a comment and a team member can reopen this issue. Opening a new issue is also acceptable!

[lolgesten]

This issue should remain open until a team member definitely tells us the current behavior is the correct one. And i seriously dislike the stale bot.

[danielpclark]

Thanks for opening this issue.

I have a library named Rutie and I've found that Rust isn't acknowledging linker paths on Rust but it is on Mac… see this issue I opened https://github.com/rust-lang/rust/issues/57550

In the meantime I either symlink or copy the Ruby dynamic library to the target/<profile>/deps dir to get it to work. But for projects that depend on Rutie they once against raise an error about loading shared libraries:

error while loading shared libraries: libruby.so.2.6: cannot open shared object file: No such file or directory

Following your advice here; setting LD_LIBRARY_PATH for that individual project that depends on Rutie to the path where Ruby's dynamic library makes it work. I.e. I can run cargo test.

So now I'm going to experiment in how to make this behavior work with Rutie automatically without requiring user's of my crate to go through this same process.

[thomwiggers]

I've run into this issue where cargo test --doc does not set LD_LIBRARY_PATH correctly, but my crate is building that shared library using cmake and it's located in the target/debug/build/.../out directory.

If you want to try this, it's this commit: https://github.com/open-quantum-safe/liboqs-rust/pull/25/commits/ef058873bc6866251f69c0823e2f983bc9ac6d8c

cargo test --doc

fails;

LD_LIBRARY_PATH=/home/thom/.../oqs-rs/target/debug/build/oqs-sys-512f6110a29d60ba/out/build/lib/:/usr/local/lib cargo test --doc

works.