llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
28.66k stars 11.85k forks source link

-path-equivalence doesn’t map path in coverage data to different path on same machine #62689

Open FlashSheridan opened 1 year ago

FlashSheridan commented 1 year ago

Our use case for the -path-equivalence option to llvm-cov seems to be covered by the descriptive part of its command documentation, “Map the paths in the coverage data to local source file paths,” though it differs from the subsequent motivational sentence (“This allows you to generate the coverage data on one machine, and then use llvm-cov on a different machine where you have the same files on a different path.”) Our details are quite complicated (involving Rust and Cargo dependencies with duplicated files), but the problem reproduces with the natural extension of the basic example in the Source-based Code Coverage: The -path-equivalence option makes no difference to the output. While there are tests which use the option, e.g. llvm/test/tools/llvm-cov/showExpansions.cpp and llvm/test/tools/llvm-cov/showProjectSummary.cpp, their checks don’t seem to depend on the option making a difference — and when imitating the tests manually, the option made no difference.

Steps to Reproduce

Here are the simplified steps, using the foo.cc from the documentation:

In directory Path_Equivalence, then repeated in the directory Other_Path_Equivalence:

/Users/flash/Documents/Code/external_code/ToT_LLVM/llvm-project/build/bin/clang++ -fprofile-instr-generate -fcoverage-mapping --sysroot=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk/    foo.cc -o foo

In directory Path_Equivalence:

LLVM_PROFILE_FILE="foo.profraw" ./foo

In directory Other_Path_Equivalence:

LLVM_PROFILE_FILE="foo2.profraw" ./foo

Back in directory Path_Equivalence:

/Users/flash/Documents/Code/external_code/ToT_LLVM/llvm-project/build/bin/llvm-profdata merge -sparse foo.profraw  ../Other_Path_Equivalence/foo2.profraw -o foo+2.profdata
/Users/flash/Documents/Code/external_code/ToT_LLVM/llvm-project/build/bin/llvm-cov show  -object ./foo -object  ../Other_Path_Equivalence/foo -instr-profile=foo+2.profdata -path-equivalence=/Users/flash/Documents/Code/Test_code/Path_Equivalence,/Users/flash/Documents/Code/Test_code/Other_Path_Equivalence

Actual Output

The output shows both files; it does not map the path …/Path_Equivalence in the coverage data to the local source file path …/Other_Path_Equivalence:

/Users/flash/Documents/Code/Test_code/Other_Path_Equivalence/foo.cc:
    1|     40|#define BAR(x) ((x) || (x))
…
/Users/flash/Documents/Code/Test_code/Path_Equivalence/foo.cc:
    1|     40|#define BAR(x) ((x) || (x))

Expected Output

We need the coverage data for both files mapped to source file path Other_Path_Equivalence/foo.cc. Obviously in this simplified example it makes no difference, but in our case the majority of our test coverage is in one dependency copy of the file(s) of interest, with some test coverage is in the original location.

Configuration

macOS 12.6.5 21G531 on M1 Max
 Clang built from source 54fd5cf926c56d255c4757af838edbe37e6ae5d9, May 12 09:31:04 2023 -0700.

opajonk commented 6 months ago

FYI I think we're running into the same issue. Basically, the -path-equivalence does not do anything...

$ llvm-cov-15 --version
Ubuntu LLVM version 15.0.7
  Optimized build.
  Default target: x86_64-pc-linux-gnu
  Host CPU: tigerlake