flub / testdir

Semi-persistent, scoped test directories
Apache License 2.0
8 stars 3 forks source link

Testdir collision for doctests #11

Open wiktor-k opened 1 week ago

wiktor-k commented 1 week ago

Hi,

I believe this may be out of scope for testdir but I think it still needs to be documented as it surprised me.

testdir::testdir!() generates the same directories when executed in a doctest.

Minimal reproducer (src/lib.rs):

/// Does one thing well.
///
/// ```
/// let dir = testdir::testdir!();
/// eprintln!("One: {dir:?}");
/// assert!(false);
/// ```
pub fn one() {}

/// Does another thing well.
///
/// ```
/// let dir = testdir::testdir!();
/// eprintln!("Two: {dir:?}");
/// assert!(false);
/// ```
pub fn two() {}

Running it with cargo test shows this output:

tderr:
Two: "/home/wiktor/tmp/cargo/testdir-198/rust_out/h9f5ab6561c35996e"
thread 'main' panicked at src/lib.rs:5:1:
assertion failed: false
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- src/lib.rs - one (line 3) stdout ----
Test executable failed (exit status: 101).

stderr:
One: "/home/wiktor/tmp/cargo/testdir-198/rust_out/h9f5ab6561c35996e"
thread 'main' panicked at src/lib.rs:5:1:
assertion failed: false
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Both of them generated the same path: "/home/wiktor/tmp/cargo/testdir-198/rust_out/h9f5ab6561c35996e" and some test may want to use isolated dirs.

This is similar to https://github.com/flub/testdir/issues/6 but instead of rstest it's doctests.

flub commented 3 days ago

I think this is fair to consider in-scope, thanks for reporting.

I think the entire directory structure created here is not intuitive. Ideally this would indicate it is a doc test and still have the module structure somehow. Not sure if this is possible though.