Low performance in a large codebase and multi-crate workspace

[blaind]

Thank you for the excellent library.

I'm encountering an issue when running individual crate tests in a large codebase, and it's causing significant initial slowness. For instance, when working with the Bevy engine repository (https://github.com/bevyengine/bevy/), running tests only in the crates/bevy_app crate necessitates building all the examples, resulting in a target directory size of over 150GB and a substantial time investment. I've managed to improve the situation to some extent by adding an args = { "--lib" } flag, but the build process remains quite slow due to the workspace flag.

The root cause appears to be related to issue #27 and commit c8894d8.

Here are some statistics below, each run starting with a clean target folder.

Current neotest-rust flags (--workspace): very very slow

crates/bevy_app$ cargo nextest run --workspace app::
time taken: [very long time]
target folder size: > 100GB

Without workspace, with --lib: very fast

crates/bevy_app$ cargo nextest run --lib app:
Finished test [unoptimized + debuginfo] target(s) in 17.72s
target folder size: 496MB

No arguments

crates/bevy_app$ cargo nextest run
Finished test [unoptimized + debuginfo] target(s) in 17.14s
target folder size: 519MB

With --lib and --workspace: somewhat reasonable, but still slow

crates/bevy_app$ cargo nextest run --lib --workspace app::
Finished test [unoptimized + debuginfo] target(s) in 1m 36s
target folder size: 8.3GB

[rouge8]

Do you have any suggestions for addressing this? You correctly identified https://github.com/rouge8/neotest-rust/commit/c8894d826a2cb5003cb6bc0a2fa8f7a359899130 as the culprit. Is there a way to make that logic conditional? Or detect the correct package/binary and use that?

[rathod-sahaab]

@rouge8 I think we should try, restricting the scope always. Some crates compilation to fail when testing so I can't use workspace test.

VScode rust-analyzer plugin does this.

cargo test --package clipper --lib -- tests --nocapture 
cargo test --package clipper --lib -- tests::test_punctuation --exact --nocapture

Need to look into neotest context for this, will update later.

@blaind Will this work for you as well?

[rouge8]

I believe that doesn’t work for non-default packages in a workspace as described in the commit linked above.