Consider better testing strategy.

[ehuss]

The current implementation has a simple set of tests. It is a bit hacky for a few reasons:

• It must share the global cargo home to download the dependencies.
• It runs in a single #[test] because it wants to reuse the artifacts because they take a very long time to build.
• It is very slow.
• It could probably be more extensive, though at the moment I'm not sure what to add.
• It does not verify that the correct libraries are actually linked.
• Alex suggested creating a "mock" standard library that reexports what's needed. However, when looking into this I didn't see a way to make it work without making a completely separate code path. I like the idea, though, so it may be interesting to investigate more.

We may want to consider moving these tests to a separate integration test so they can be separated and do whatever special requirements they need. However, this would require sharing the support module somehow (either as a separate crate or just build it twice).

[alexcrichton]

I poked around at the mock idea today and I think it might actually work. I'm heading out for today but I've got a WIP at https://gist.github.com/alexcrichton/3b8385d6ad8e487d4e6942a763d9f4d7 (sorry weird format, didn't want to make a temp repo for this) where the crux is that libcore prints out -L /path/to/the/real/sysroot and then after that we kinda just make sure everything wires up right.

The only diff needed in Cargo to use this was:

diff --git a/src/cargo/core/compiler/standard_lib.rs b/src/cargo/core/compiler/standard_lib.rs
index fb58363f0..af8b86068 100644
--- a/src/cargo/core/compiler/standard_lib.rs
+++ b/src/cargo/core/compiler/standard_lib.rs
@@ -148,6 +148,10 @@ pub fn generate_std_roots<'a>(
 }

 fn detect_sysroot_src_path(ws: &Workspace<'_>) -> CargoResult<PathBuf> {
+    if true {
+        return Ok("/home/alex/code/mock-std".into())
+    }
+
     // NOTE: This is temporary until we figure out how to acquire the source.
     // If we decide to keep the sysroot probe, then BuildConfig will need to
     // be restructured so that the TargetInfo is created earlier and passed

I'm envisioning that we'd check in this mock tree into Cargo itself and then all tests would just use it, and every test would "rebuild std" but it's just a bunch of empty path dependencies so we're talking about a hundred or so ms vs the multi seconds we have today! Ideally we could soup this up over time as well and have our "sysroot crates" depend on "crates.io crates" and make sure that all the stars align.

This is enough mocking out that I still think we want at least one dedicated test which does the whole end-to-end shebang, but I'm hoping we can write 95% of the tests against a "mock" sysroot which compiles quickly so we can continue to run things in parallel with standard #[cargo_test]

[alexcrichton]

Fixed in above PR