weihanglo
commented
2 years ago Firstly, in cargo each target compiles down to a crate, which usually refers to a compilation artifact such like a library or a executable.
For unit test, cargo invokes rustc to collect all #[test] items and link them to libtest for building final executable. This executable runs all your unit tests inside a target and by default in parallel unless you pass cargo test -- --test-threads 1. For instance, if you have one lib target and one bin. When you run cargo test, Cargo produces two unittest executables: One for lib and the other for bin. The two executables are run one by one, just like how cargo-nextest describes in its own book. They are two processes spawn by cargo, so they might inherit a part of environment from cargo. But all #[test] inside the same target are just run under the same process. You can consult the relevant code here for more info.
The aforementioned rule also applies on integration tests, all #[test] items inside on test target are compiled into on libtest executable. Each integration test produces a crate so that each is run in different process. This seems somehow inefficient and people sometimes combine all integration tests into one target to exploit libtest's parallelism. (cargo itself does that as well)
Things get interesting in doc tests. Cargo permits only one lib target for each package, and only compiles doc test for lib target. You might ask, does that implies a package gets exactly one executable for running doc tests? The answer is No. Doc tests current are controlled by rustdoc, which leverages library artifact such like rlib to compile each code block into an executable respectively. So, instead of one executable, you get multiple exetuables if you have multiple code blocks to test. These exectuables are located in a temporary directory by default, so you won't see them in the target dir. The relevant code lies in here. Interestingly, code blocks of doctest are run in parallel although they are different executables.
Here is a table of my summary:
| Unit tests | Integration tests | Doctests | |
|---|---|---|---|
| Run in parallel? | ✅ for #[test] ❌ for target |
✅ for #[test] ❌ for target |
✅ for each code block |
| Share environment? | One target per process | One target per process | One code block per process |
My knowledge might be rusty. Correct me if you find anything incorrect.
Hi. I aware of 3 types of tests in Rust: unit tests, integration tests and doctests. I would like to see in documentation for
cargo testin Cargo book and in Rustdoc book the following info: whether this tests run in parallel and whether every test is run in fresh environment, i. e. in new process created specially for this test. Please note that "sequential execution" and "execution of each test in fresh environment" are different things. Consider unit tests run usingcargo test -- --test-threads=1. As well as I understand they will be run sequentially, but they still be run in the same process, thuschdirand similar operations from one test will affect others. I want to see in cargo book and other docs answers to this questions:#[test]functions from same file intests) run in parallel?#[test]functions from same file intests) run in fresh environment?testsrun in parallel?tests) run in fresh environment?Currently I was able to easily find answers to first two questions. But docs are vague on others. Section https://doc.rust-lang.org/rustdoc/documentation-tests.html doesn't answer questions on doctests
(moved from https://github.com/rust-lang/book/issues/3042 and slightly edited)