cdylib / staticlib libraries aren't created in the target directory when running `cargo test`

[jgallagher-cs]

Problem

I'm not 100% sure this is a bug, and as an attempt to avoid an X/Y problem, I'll start by describing what I'm ultimately trying to do (which is related to #7825).

I have Rust crates with crate-type = ["cdylib"] or crate-type = ["staticlib"] (or both), and I'd like to run what are essentially integration tests written in C to ensure that my header files are correct, the libraries are built correctly, etc. And I'd really like for those tests to run when I run cargo test.

What I'm currently doing which almost works is that I have two crates: one that produces the C library and the other that exists solely to run the integration tests. The testing crate uses cc to build C code that calls the library produced by the "real" crate; its build.rs looks roughly like this (this example is assuming a staticlib):

fn main() {
    cc::Build::new()
        .file("src/tests.c")
        .include("../real-crate/include")
        .compile("c_integration_tests");

    let profile_dir = /* ... */; // find "target/release" or "target/debug" based on OUT_DIR
    println!("cargo:rustc-link-search=native={}", profile_dir);
    println!("cargo:rustc-link-lib=static=real_crate");
}

There are (at least) two problems with this:

• Finding target/release or target/debug based on OUT_DIR is pretty hacky.
• target/release/libreal_crate.a isn't actually built when I run cargo test (even if it's a dependency); only the hash-suffixed library name (target/release/deps/libreal_crate-XXXXXXXXXXXXXXX.a) exists.

The first problem is a little gross but hasn't seemed to be a big deal in practice, although I assume it's something that might change over time. The second is more problematic; a workaround is to cargo build before cargo testing, but it's hard to remember to do that every time, and it's not ideal to have to tell everyone working on the project that that's a requirement.

Steps

1. Create a crate with crate-type = ["staticlib"].
2. Create another crate that depends on it.
3. Run cargo test.
4. The static lib from the first crate is not built in the target dir.

Possible Solution(s)

I considered finding target/{release,debug}/deps via OUT_DIR then scanning that directory to find the library-with-suffix and linking that, but that seems even more fragile.

Notes

Output of cargo version:

cargo 1.43.0 (3532cf738 2020-03-17)

[joshtriplett]

This makes sense. In general, there's a use case for depending on a crate that provides a cdylib, a staticlib, or even a bin, and then being able to use the resulting artifact at build time.

We discussed this in the Cargo meeting today.

I'd propose the following (sketch of a) solution:

• Cargo.toml should allow specifying dependencies (or build dependencies, or dev dependencies) on a staticlib or cdylib or bin.
  • This likely will require some additional indication of what type(s) of artifact the dependency wants.
  • We need to carefully think about whether the type of dependency fully specifies whether you want it built for the host or the target, or if we need an explicit way to specify this.
  • We need to consider the possibility of allowing flexible selection between static and dynamic libraries, if a crate can support building either or both.
• Given such a dependency, Cargo will build the artifacts you asked for, in your output directory.
• Cargo will then export environment variables pointing to those artifacts, along the lines of CARGO_DEP_<depname>_BIN, CARGO_DEP_<depname>_STATICLIB, or CARGO_DEP_<depname>_CDYLIB.

[jgallagher-cs]

Cool! A couple thoughts on your solution sketch:

• Does the "caller" need to specify which kind of artifact(s) it wants, or should that be pulled from the dependency itself? At a minimum, the caller can't ask for a cdylib or staticlib if those aren't in the dependency's crate-type, right?
• I hadn't considered depending on bins but what if the dependency has more than one?

I don't have any experience with cross compilation, so can't comment on the host vs target question.

[asomers]

I'm experiencing this exact same problem. I want to write an integration test for a cdylib. I can guess the built file's name easily enough, but I can't figure out how to force cargo to build it before (or while) running the test. Is there any better workaround than "run cargo build before cargo test"?

[dkg]

Any word on this? I'd like to be able to automate these kinds of integration tests in a cdylib crate as well (see https://github.com/integritychain/fips203/pull/13)