Add cargo:rustc-dynamic-link-search for setting LD_LIBRARY_PATH during cargo run/cargo test

[Manishearth]

Currently when building we tell cargo where to find static and shared libraries with cargo:rustc-link-search.

However, if you call cargo run or cargo test, while it will still compile, it won't run because LD_LIBRARY_PATH or DYLD_LIBRARY_PATH need to be set. If crates could explicitly emit dylib search paths with cargo:rustc-dynamic-link-search then we could print a list of them at the end of compilation (the same way we do for staticlibs, something like "run this with LD_LIBRARY_PATH=foo:bar:baz"), and if folks use cargo run or cargo test we can run those with this appended to the dynamic lib path

[mgattozzi]

Would it be possible to have this for libraries as well? So if I do a cargo build or if someone uses my library it'll know how to have this set in case they use it? Or would that be part of cargo run where the library is eventually used?

[Manishearth]

Right, that's the proposal.

[Michel-Haber]

@Manishearth Anything on this? It does look rather necessary for consistency that a "cargo run" or "cargo test" not crash simply because it has dynamic dependencies, especially after a successful build. This is problematic when a user cannot, or does not want to put these libraries in the default lib path. If it remains imperative that the purview of cargo be the target folder, it might be a valid solution for cargo to copy the libs into the target folder. This however can spell trouble if these libs are too large.

[Manishearth]

This isn't my decision to make, cc @alexcrichton

[alexcrichton]

I don't really understand the problem nor the proposal here, so I can't really comment much on this.

[Manishearth]

You may have a build script that builds or pulls in a dylib. For native libraries (static and dynamic), this needs a corresponding -L flag, which gets provided by emitting cargo:rustc-link-search.

This all works fine. You can compile things that build dylibs on the fly.

However, you cannot cargo run or cargo test them since they don't know where to look for the dylib at runtime! I'm proposing a cargo:rustc-dynamic-link-search flag whose output is stored somewhere and picked up by cargo run/cargo test to be used to set LD_LIBRARY_PATH.

[alexcrichton]

Cargo should already modify the dynamic library search path for anything in the build directory, but if the path is on the system it's up to the user to configure that to be used

[Manishearth]

Yeah, sorry, this isn't exactly for build script ones, it could be where the build script needs to do some config to search for a dylib on the system.

We've been over this before: https://github.com/rust-lang/cargo/issues/4887#issuecomment-355044354

You suggested opening a feature request for specifically instructing cargo to do this. This is said feature request :smile:

[alexcrichton]

Ah sorry, my mistake! This definitely still stands as a feature request and makes sense to me to implement!

[danielpclark]

I really want this feature. My project Rutie depends on a library which I kind of cheat to get to working during its own build. But when other projects use it the dynamic library is no longer found so cargo test fails. See comment for more detail: https://github.com/rust-lang/cargo/issues/4044#issuecomment-498508207

[Lord-Kamina]

Please use DYLD_FALLBACK_LIBRARY_PATH instead of DYLD_LIBRARY_PATH. The latter is a hack and almost certain to break things for most people running macOS.

_{Sent with GitHawk}

[mxxo]

This would be a very helpful feature for wrapping C libraries that only ship shared objects.

Some scientific computing crates are like this.

• The wrapper around the GNU Scientific library https://github.com/GuillaumeGomez/rust-GSL requires the gsl library.

• Shameless plug: some Gmsh library bindings I am working on https://github.com/mxxo/gmsh-rs

Especially for users without root access, specifying the directory to find dynamic libraries (e.g. ~/.local/lib) would be really great.

[llogiq]

I just came upon the exact same problem in a project that uses ndarray-linalg and thus blas-src. Now our tests failed on CI because it wouldn't dynamically load netlib or openblas, whichever we tried. Adding the library paths manually or installing them in /usr/lib would fix the problem, but the former seems like a bit of a hack and the latter may be a problem for deployment later.

[problame]

FWIW I found this stackoverflow post that recommends setting "cargo:rustc-env=LD_LIBRARY_PATH from build.rs. Works for me on cargo 1.49.0 (d00d64df9 2020-12-05). My use case is building a Rust cdylib that links against another .so in a bigger C project.

[christopinka]

I'm unclear why there isn't an option to move the dynamic library under target/debug/deps if you want to treat it like a dependency. Is size the main consideration?

[christopinka]

for me on cargo 1.49.0 (d00d64df9 2020-12-05). My use case is building a Rust cdylib that links against another .so in a bigger C project.

The answer uses an absolute path. I don't think that's a great solution.

[teor2345]

Just ran into this issue today while trying to get CUDA libraries running on a non-standard Linux distribution.

It’s weird that cargo run/test replaces the LD_LIBRARY_PATH rather than prepending to it. But we might have to live with that behaviour now.

A workaround is a build script with something like:

if let Ok(path) = std::env::var("PRE_CARGO_LD_LIBRARY_PATH") {
    println!("cargo:rustc-env=LD_LIBRARY_PATH={path}");
}

Edit: my original suggestion didn't work because cargo has already replaced LD_LIBRARY_PATH by the time the build script runs.