search

taiki-e / cargo-llvm-cov

Cargo subcommand to easily use LLVM source-based code coverage (-C instrument-coverage).
Apache License 2.0
933 stars 57 forks source link

Duplicated methods in lcov report cannot be merged #382

Closed gth828r closed 2 months ago

gth828r commented 2 months ago

Due to how tests can be structured and how the compiler mangles function names, it is possible to have a function defined multiple times in the lcov.info with different names. This leads to the file also including duplicate coverage data which can be conflicting depending on the test.

I believe this was addressed in the lcov project with a command line parameter --filter function: https://github.com/linux-test-project/lcov/pull/86/commits/d336b687d0124b183cf266db3530b7303e70477e

As far as I can tell, that parameter is not exposed via cargo-llvm-cov. Can that parameter be exposed?

gth828r commented 2 months ago

Here is an example with some irrelevant bits out:

SF:/resources/peregrinator-nrt-ric/xapp_runner/src/runner.rs
FN:238,_RNCNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s_00CshdvICJUe1WP_17integration_tests
 <snipped>
FN:212,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run00CshdvICJUe1WP_17integration_tests
FN:257,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s0_0CshdvICJUe1WP_17integration_tests
FN:285,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s1_0CshdvICJUe1WP_17integration_tests
FN:246,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s_0CshdvICJUe1WP_17integration_tests
 <snipped>
FN:171,_RNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0CshdvICJUe1WP_17integration_tests
 <snipped>
FN:171,_RNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0B5_
FN:212,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run00B7_
FN:246,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s_0B7_
FN:238,_RNCNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s_00B9_
FN:257,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s0_0B7_
FN:285,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s1_0B7_
 <snipped>
FN:167,_RNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run
 <snipped>
FN:167,_RNvNtCsdLA1neVDAz0_24peregrinator_xapp_runner6runner3run

Note that the source file did not change across these records, and there are duplicate entries for the line number where the mangled function name differs. In my case, I have a project with multiple workspaces, and one of those workspaces includes an end-to-end integration test that exercises code across all of the workspaces. Only the integration test is hitting the particular function I am highlighting in these snippets.

The reported function coverage looks like this:

FNDA:1,_RNCNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s_00CshdvICJUe1WP_17integration_tests
FNDA:1,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run00CshdvICJUe1WP_17integration_tests
FNDA:1,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s0_0CshdvICJUe1WP_17integration_tests
FNDA:1,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s1_0CshdvICJUe1WP_17integration_tests
FNDA:1,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s_0CshdvICJUe1WP_17integration_tests
 <snipped>
FNDA:1,_RNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0CshdvICJUe1WP_17integration_tests
FNDA:1,_RNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner8run_xapp0CshdvICJUe1WP_17integration_tests
FNDA:1,_RNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner8shutdown0CshdvICJUe1WP_17integration_tests
 <snipped>
FNDA:0,_RNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0B5_
FNDA:0,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run00B7_
FNDA:0,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s_0B7_
FNDA:0,_RNCNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s_00B9_
FNDA:0,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s0_0B7_
FNDA:0,_RNCNCNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run0s1_0B7_
 <snipped>
FNDA:1,_RNvNtCsgkPIuEin8tU_24peregrinator_xapp_runner6runner3run
 <snipped>
FNDA:0,_RNvNtCsdLA1neVDAz0_24peregrinator_xapp_runner6runner3run
FNDA:0,_RNCNvNtCsdLA1neVDAz0_24peregrinator_xapp_runner6runner3run0B5_
FNDA:0,_RNCNCNvNtCsdLA1neVDAz0_24peregrinator_xapp_runner6runner3run00B7_
FNDA:0,_RNCNCNvNtCsdLA1neVDAz0_24peregrinator_xapp_runner6runner3run0s_0B7_
FNDA:0,_RNCNCNCNvNtCsdLA1neVDAz0_24peregrinator_xapp_runner6runner3run0s_00B9_
FNDA:0,_RNCNCNvNtCsdLA1neVDAz0_24peregrinator_xapp_runner6runner3run0s0_0B7_
FNDA:0,_RNCNCNvNtCsdLA1neVDAz0_24peregrinator_xapp_runner6runner3run0s1_0B7_

In a nutshell, some of the duplicate functions report coverage, while others do not. In my case, this is a problem for the VS code Coverage Gutters extension, which seems to be looking at only the functions which report no coverage at all.

Note that this is not an issue for the standard summary report that is provided by cargo-llvm-cov. It is specific to the lcov output and the downstream tools that consume it.

taiki-e commented 2 months ago

Same issue as https://github.com/taiki-e/cargo-llvm-cov/issues/325#issuecomment-2252056955?

gth828r commented 2 months ago

Same issue as #325 (comment)?

It sure seems like it. Sorry, the subject line for that issue didn't catch my attention when I searched before creating my issue. Let me try rustfilt and see if it addresses my issue.

gth828r commented 2 months ago

It seems that upgrading to version v0.6.7+ for the new demangling features as well as using rustfilt is not sufficient to address this issue. The demangling works just fine, but it does not actually merge any entries in the lcov.info file. The result is that now, there are exact duplicate FN: entries in the file, since the demangling works, but there are still multiple FNDA: entries corresponding to the duplicate functions, and some of those entries show hits while the others do not.

I am not sure how coverage gutters works. I will try playing around with removing these duplicates manually and see if I can get it to show which lines are properly covered and which are not.

gth828r commented 2 months ago

I have tried removing the FN and FNDA duplicate entries from the lcov.info file (demangled by rustfilt) for the source file I am testing with. Coverage Gutters is still reporting 0% coverage and showing no lines marked as covered. I guess I am still not sure what the underlying issue is now that manually removing duplicate entries did not fix the issue. For now, I am going to close this out since I don't have any proposed action items related to cargo-llvm-cov anymore. I will re-open with a new issue if I find something.

Thanks for your support!

gth828r commented 2 months ago

Apologies, I found my issue -- there were some stale lcov.info files which had been generated deeper in my source tree, and coverage gutters was finding those and using them. After removing those files, things work as expected with duplicate functions with different mangled names and different coverage count metrics associated with those different lines being reported. I verified that things work with a version prior to v0.6.7 and a version after v0.6.7, so the features to deal with mangling mentioned in https://github.com/taiki-e/cargo-llvm-cov/issues/325#issuecomment-2252056955 seem unrelated.