search

nextest-rs / nextest

A next-generation test runner for Rust.
https://nexte.st
Apache License 2.0
2.11k stars 92 forks source link

[windows] lots of tests running in parallel may cause "leaky" warnings #1469

Open zaneduffield opened 4 months ago

zaneduffield commented 4 months ago

When I run my tests with datatest-stable (0.2.9), using nextest as the runner, a semi-random number of tests are marked as leaky.

This seems to only depend on how many tests the crate has, and not what the tests are doing.

I have an example repo here: https://github.com/zaneduffield/datatest_leak_issue That repo has 1000 test files, which are used with a no-op test function. When I run cargo nextest run on it, about 40% of the tests are marked as 'leaky'. One issue that may be related is how slow this is: ~17s on my machine compared to ~0.2s with cargo test.

zaneduffield commented 4 months ago

I should mention that I am running this on Windows.

zaneduffield commented 4 months ago

Maybe this is related to AV getting in the way (https://nexte.st/book/antivirus-gatekeeper). Even if that is the case, I'm not sure why the tests would be considered leaky.

sunshowers commented 4 months ago

Thanks for the report. It does seem like antivirus getting in the way. Have you tried the new "dev drive" feature on Windows?

Leak detection works by monitoring the stdout and stderr file descriptors after the file has closed. If antivirus or something else is keeping the fds open, this issue will be hit. You can bump up the timeout with the "leak-timeout" parameter.

I think this isn't really a datatest-stable issue, more of a nextest issue. Moving it there.