Closed mssun closed 3 years ago
Do you have any examples of tests using assert_cli
where tarpaulin doesn't work? Just because that's the first step in supporting this. Also, tarpaulin should follow forks down so is there something more going on in assert_cli
?
Sure.
Using an hello_world
app as an example:
First crate the app:
cargo new hello_world --bin
Add a integration test using assert_cli
:
$ cat src/main.rs
fn main() {
println!("Hello, world!");
}
$ cat tests/tests.rs
extern crate assert_cli;
#[test]
fn test_main() {
assert_cli::Assert::main_binary().succeeds().stdout().is("Hello, world!\n").unwrap();
}
$ cat Cargo.toml
[package]
name = "hello_world"
version = "0.1.0"
authors = ["Bob Sun <bob@mssun.me>"]
[dependencies]
[dev-dependencies]
assert_cli = "0.5"
Here is the result of tarpaulin:
cargo tarpaulin
Building project
Launching test
running /ssd/mssun/hello_world/target/debug/deps/hello_world-d8edb80c0ec19daa
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
Launching test
running /ssd/mssun/hello_world/target/debug/deps/tests-db878f2c6d8489d2
running 1 test
.
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
Coverage Results
src/main.rs: 0/2
tests/tests.rs: 2/2
50.00% coverage, 2/4 lines covered
Tarpaulin finished
The src/main.rs
should be covered by the test case, but it is not showed in the result.
This is definitely due to cargo run. Part of me hoped it would be as easy as removing the --release
flag since that would cause an issue but the same thing happened. I'll have a look into it but it might take a while till I have anything, there are a lot of moving parts.
It looks like tarpaulin doesn't support multiprocess tracing. When a test execs another process, tarpaulin detaches ptrace from it. Other parts of the code only focus on single-process tracing. For example, LinuxData
has a current
field that only tracks one PID.
I did try and do some work on this a month ago or so which I summarised some things from here https://github.com/xd009642/tarpaulin/issues/107 and started here https://github.com/xd009642/tarpaulin/tree/follow_spawned_processes though I haven't pushed anything apart from a test.
Party of the problem I have to tackle is when a process is launched I need to stop it, get it's executable name and figure out if it's something I can instrument? (Is it in the target folder), instrument it and then potentially handle both two machines.
If this involves fork it won't work yet, but if it just executes another process I should have a working prototype of following process exec's on https://github.com/xd009642/tarpaulin/pull/615 if you want to try it out. So far I've just tried it on a very simple toy project so more people testing it is appreciated 😄. Adding fork support should be trivial and a next step
So I don't yet follow fork children. But I follow fork parents and execed processes now. I tried on assert_cmd and got some coverage but I did notice fork events so may not quite be as thorough as you'd want. I'm just going to close this in favour of a more specific issue for forks as they're the current missing piece. https://github.com/xd009642/tarpaulin/issues/616
Anyone finding this issue via search. Use --follow-exec
to enable it.
Any chance to support CLI app's integration tests? For example,
tarpaulin
won't calculate coverage when usingassert_cli
https://github.com/assert-rs/assert_cli to fork and execute CLI and test. Thanks.Related discussion: https://github.com/rust-lang-nursery/cli-wg/issues/9#issuecomment-385487756