search

xd009642 / tarpaulin

A code coverage tool for Rust projects
https://crates.io/crates/cargo-tarpaulin
Apache License 2.0
2.5k stars 180 forks source link

Doc tests are not being run #850

Closed danieleades closed 2 years ago

danieleades commented 3 years ago

Describe the bug Tarpaulin is failing to run doc tests in one of my projects

To Reproduce project is here - https://github.com/danieleades/rtps

you can also see the github action output here - https://github.com/danieleades/rtps/runs/3850610019?check_suite_focus=true

Expected behavior doc tests should be run and produce non-zero coverage

xd009642 commented 3 years ago

So doc test coverage isn't ran by default you need to run --doc. I'm not sure in the long term, if I want to change the default behaviour as most people don't track coverage of doctests. In the short term, this won't be changing as doctest coverage only works on nightly because it requires the unstable --persist-doctests flag: https://github.com/rust-lang/rust/issues/56925 and I'd rather not confuse people with their coverage stats potentially changing dramatically when they switch between different toolchains

danieleades commented 3 years ago

that makes sense. thanks for the explanation

danieleades commented 3 years ago

hang on a tic, if I use the --doc flag, then only the doctests are run. if i use the --tests flag, then the doc tests are excluded. Is there a way to run all of the unit tests, integration tests, and doc tests at once?

xd009642 commented 3 years ago

the test-type flags are collected into a list so you can do cargo tarpaulin --doc --tests

danieleades commented 3 years ago

the test-type flags are collected into a list so you can do cargo tarpaulin --doc --tests

Pretty sure I tried that, and it complained... Will double-check when I get a chance

xd009642 commented 3 years ago

I double checked it on the all_test_types test project in the repo and it worked as expected, so hopefully it will work for you :crossed_fingers: 

$ cargo tarpaulin --version
cargo-tarpaulin version: 0.18.2
$ cargo +nightly tarpaulin --doc --tests
Oct 10 12:01:33.749  INFO cargo_tarpaulin::config: Creating config
Oct 10 12:01:34.156  INFO cargo_tarpaulin: Running Tarpaulin
Oct 10 12:01:34.156  INFO cargo_tarpaulin: Building project
Oct 10 12:01:34.156  INFO cargo_tarpaulin::cargo: Cleaning project
   Compiling all_test_types v0.1.0 (/home/daniel/personal/tarpaulin/tests/data/all_test_types)
    Finished test [unoptimized + debuginfo] target(s) in 0.42s
   Doc-tests all_test_types
   Compiling all_test_types v0.1.0 (/home/daniel/personal/tarpaulin/tests/data/all_test_types)
    Finished test [unoptimized + debuginfo] target(s) in 0.39s
Oct 10 12:01:35.340  INFO cargo_tarpaulin::process_handling::linux: Launching test
Oct 10 12:01:35.340  INFO cargo_tarpaulin::process_handling: running /home/daniel/personal/tarpaulin/tests/data/all_test_types/target/doctests/src_only_doc_test_rs_2_0/rust_out
Oct 10 12:01:35.627  INFO cargo_tarpaulin::process_handling::linux: Launching test
Oct 10 12:01:35.627  INFO cargo_tarpaulin::process_handling: running /home/daniel/personal/tarpaulin/tests/data/all_test_types/target/debug/deps/all_test_types-bc1fd442f8ad3c26

running 1 test
test only_test::tests::test_it ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

Oct 10 12:01:35.952  INFO cargo_tarpaulin::report: Coverage Results:
|| Tested/Total Lines:
|| src/only_bench.rs: 0/3 +0.00%
|| src/only_doc_test.rs: 3/3 +100.00%
|| src/only_example.rs: 0/3 -100.00%
|| src/only_test.rs: 5/5 +0.00%
|| 
57.14% coverage, 8/14 lines covered, -9.523809523809518% change in coverage
xd009642 commented 2 years ago

@danieleades did you have a chance to check this?

danieleades commented 2 years ago

@danieleades did you have a chance to check this?

I did! Works like a charm. Suspect I fat-fingered it on the first attempt.

Thankyou very much for the support