Closed Dushistov closed 1 year ago
cargo-llvm-cov limits the parallelism in test binaries to avoid the rustc bug. IIRC cargo-nextest does not respect RUST_TEST_THREADS
(https://github.com/nextest-rs/nextest/issues/652), so in cargo llvm-cov nextest
this is controlled using NEXTEST_TEST_THREADS
environment variable. However, NEXTEST_TEST_THREADS
seems to also limit overall parallelism.
@sunshowers: Is there a way to control only the number of threads created by test binaries without affecting the overall parallelism? (In other words, is there an equivalent option to RUST_TEST_THREADS
?)
@taiki-e I think RUST_TEST_THREADS
should work for that -- nextest doesn't do anything with it, and I think RUST_TEST_THREADS
is read by libtest.
Hmm, at the time of https://github.com/nextest-rs/nextest/issues/652 reported, we already set RUST_TEST_THREADS=1
, but it did not seem to be enough to avoid the rustc bug.
In that case I think something in rustc or llvm-cov might not support multiple test binaries being run in parallel.
So, may be just pass -C llvm-args=--instrprof-atomic-counter-update-all
,
and remove setting of RUST_TEST_THREADS
to 1
?
Yes, at least RUST_TEST_THREADS=1
can be replaced by -C llvm-args=--instrprof-atomic-counter-update-all
. It is not clear yet if NEXTEST_TEST_THREADS
can be removed.
If we can understand how cargo-nextest determines the number of tests to run simultaneously, we may also avoid the need to set NEXTEST_TEST_THREADS by using %Nm.
https://clang.llvm.org/docs/SourceBasedCodeCoverage.html#running-the-instrumented-program
“%Nm” expands out to the instrumented binary’s signature. When this pattern is specified, the runtime creates a pool of N raw profiles which are used for on-line profile merging. The runtime takes care of selecting a raw profile from the pool, locking it, and updating it before the program exits. If N is not specified (i.e the pattern is “%m”), it’s assumed that N = 1. The merge pool specifier can only occur once per filename pattern.
For a program such as the Lit testing tool which invokes other programs, it may be necessary to set LLVM_PROFILE_FILE for each invocation. The pattern strings “%p” or “%Nm” may help to avoid corruption due to concurrency. Note that “%p” is also a Lit token and needs to be escaped as “%%p”.
we currnetly sets %m (i.e., %1m)
Would it make sense to add a command to nextest showing the effective config along with its source?
Hmm, it may not actually be necessary to know the exact value specified in config. Here is a todo comment I added in #279:
In crate I have "unit" tests and "functional" one.
Because of number of CPU on my machine is >=2 I expect that execution time
cargo nextest
is equal to time of the slowest test -test_30
, and it works as expected:but with
llvm-cov
it for some reasons waits all unit tests to pass, before starting functional tests:On real project all looks worse. Is any way to run functional and unit tests in parallel with llvm-cov?