Closed michaelvlach closed 1 year ago
Maybe the same issue as https://github.com/taiki-e/cargo-llvm-cov/issues/43
We are aware of the issue that the llvm-cov summary report is somewhat different from the full report, and I think it may be possible to output a more accurate summary by exporting the coverage in JSON format and then converting it (--show-missing-lines and --codecov are actually implemented in this way).
It definitely seems related. I think it's LLVM treating them same as templates in C++. They are technically not templates/generics but Rust's compilation model recompiles everything but with different settings so from LLVM's perspective they are technically different instantiations (like they were templates / generics).
And thanks for the suggestions!
--json
format has the same uncovered lines but --codecov
format does not--show-missing-lines
does not show anything uncovered (which is correct) but --fail-uncovered-lines 0
still failsI think the workaround for now is to substitute --fail-uncovered-lines
for parsing --show-missing-lines
verifying there are indeed none. Would it perhaps make sense to change the logic behind --fail-uncovered-lines
from parsing the (inaccurate) report to what --show-missing-lines
does?
Feel free to decline the PR if you disagree as I know it's not as clean cut decision here since the current logic follows (and is consistent with) the report (however inaccurate). On the other hand when it fails and the report shows uncovered lines then running with --show-missing-lines
shows nothing. So I think it would make sense making it a bit more consistent that way.
Oh, that's why not generic functions some times have several instantiation according to html generated by cargo-llvm-cov:
Is this actually solved? I’m still seeing the issue described just above with the latest llvm-cov version.
It looks to me like the solution would probably be the same as #43, but the way it manifests is different, because it shows as multiple instantiations of a single function due to different #[cfg] flags.
Does that make sense?
@Ekleog-NEAR Could you open a new issue?
What was initially reported in this issue has been fixed and what you linked to is a half off-topic comment on a different API than that.
If you think there is another issue, please open an individual issue. Addressing several different issues in a single GH issue can be confusing, and it can slow down and obstruct solving the problem. Just as I struggled in https://github.com/taiki-e/cargo-llvm-cov/issues/300#issuecomment-1688304667.
Of course! I just opened https://github.com/taiki-e/cargo-llvm-cov/issues/344 to track it :)
When running coverage against a package with both unit tests and integration tests the resulting report contains two distinct instantiations on functions. This is due to different compilation flags (unit tests being with
cfg(test)
and integration tests without). This confuses up summary line coverage calculation. Detailed report shows all lines as covered with each instantiation showing respective partial coverage. The summary however claims some lines are uncovered, specifically those covered only by unit tests as seemingly only the integration tests instantiation is taken into account for summary.Minimal reproduction:
llvm-cov-merged-report-incorrect.zip
Run
cargo llvm-cov --html --open
.Some findings:
--sparse
flag ofllvm-profdata merge
does not matter (same result with or without)This is not an issue of cargo-llvm-cov per se but rather an issue of LLVM, Rust compiler or some llvm-cov config (missing flag or somesuch). But I wonder if there is some way to work around this in cargo-llvm-cov.