Coverage with doctests

[tgross35]

Nextest unfortunately can't support doctests yet (https://github.com/nextest-rs/nextest/issues/16), but is there a way to have test coverage report include doctests?

Maybe cargo-llvm-cov can somehow combine the reports of cargo llvm-cov nextest --lcov --output-path lcov.info with the result of llvm-cov running doctests on its own.

[taiki-e]

Maybe cargo-llvm-cov can somehow combine the reports of cargo llvm-cov nextest --lcov --output-path lcov.info with the result of llvm-cov running doctests on its own.

Yes. See https://github.com/taiki-e/cargo-llvm-cov?tab=readme-ov-file#merge-coverages-generated-under-different-test-conditions (or https://github.com/taiki-e/cargo-llvm-cov?tab=readme-ov-file#get-coverage-of-external-tests) for details.

[sunshowers]

Worth documenting this at https://nexte.st/book/test-coverage.html.

[tgross35]

Awesome, thanks for the hint. I think I got this working with https://github.com/pluots/zspell/blob/045c758f15df8ace6d7a633c2f0c52d1448ff936/.github/workflows/validation-rust.yaml#L79-L93

[tgross35]

Ah maybe that doesn't work quite right. I definitely use a function in a doctest here https://app.codecov.io/gh/pluots/zspell/commit/045c758f15df8ace6d7a633c2f0c52d1448ff936/blob/zspell/src/lib.rs#L36 but it still shows up as uncovered here https://app.codecov.io/gh/pluots/zspell/commit/045c758f15df8ace6d7a633c2f0c52d1448ff936/blob/zspell/src/lib.rs#L161

[taiki-e]

Oh, that's a cargo-llvm-cov bug. It should be fixed by passing --doctests to cargo llvm-cov report with just released cargo-llvm-cov 0.5.36 (https://github.com/taiki-e/cargo-llvm-cov/commit/5289e9e0b2c41decd568de3bad63f329b83ac271).

[tgross35]

Thanks for the quick fix! Seems to work now with my CI setup at https://github.com/pluots/zspell/blob/f342249d360163c30cd5eba2bcca3af1439ba633/.github/workflows/validation-rust.yaml#L79-L93

[sunshowers]

Added documentation.