LCOV output started reporting way too small numbers (while Markdown output shows the expected ones)

hartwork commented 3 weeks ago

Hello!

At some point in time LCOV output started reporting way too small numbers while the Markdown output shows the expected ones. Here is proof and more information:

Markdown output, as expected

$ grcov --binary-path target/debug --source-dir src --ignore '/*' --branch --llvm-path /usr/bin -t markdown -o target/coverage/summary.md .
$ cat target/coverage/summary.md
| file                   | coverage | covered   | missed_lines       |
|------------------------|----------|-----------|--------------------|
| command_line_parser.rs | 100.00%  | 152 / 152 |                    |
| exec.rs                | 100.00%  | 57 / 57   |                    |
| main.rs                | 97.65%   | 208 / 213 | 38, 75, 81, 85, 99 |
| network.rs             | 100.00%  | 157 / 157 |                    |
| logging.rs             | 97.56%   | 160 / 164 | 63-65, 103         |

Total coverage: 98.79%

LCOV output, broken

$ grcov --binary-path target/debug --source-dir src --ignore '/*' --branch --llvm-path /usr/bin -t lcov -o target/coverage/lcov .
$ lcov --list target/coverage/lcov
                              |Lines       |Functions  |Branches    
Filename                      |Rate     Num|Rate    Num|Rate     Num
====================================================================
[/]
command_line_parser.rs        | 7.2%    152| 0.0%    11|    -      0
exec.rs                       |12.3%     57| 0.0%     7|    -      0
logging.rs                    |22.0%    159| 0.0%    33|    -      0
main.rs                       |21.2%    208| 0.0%    44|    -      0
network.rs                    | 7.6%    157| 0.0%    12|    -      0
====================================================================
                        Total:|14.9%    733| 0.0%   107|    -      0

Additional info

This is with both grcov 0.8.18 and 0.8.19 and 0.8.20
This is with LCOV version 2.0-4ubuntu2 and also 2.0-1
The code base for this was https://github.com/hartwork/rust-for-it/ .
There is a related bug https://github.com/hartwork/rust-for-it/issues/171 and pull request https://github.com/hartwork/rust-for-it/pull/181

Best, Sebastian

marco-c commented 2 weeks ago

@hartwork could you try with older grcov versions, so we can figure out what caused this?

hartwork commented 2 weeks ago

@marco-c I experimented with it more and found that even with 0.8.18 from back when I first started using grcov in Jun 2023 at https://github.com/hartwork/rust-for-it/pull/8 produces this issue today. So given the binary contains all the Rust dependencies from that time baked in it seems to mean that either the commands that grcov is calling out to have changed and take part in the problem (but then I'd expect markdown output to also be affected) or that the LCOV output that grcov 0.8.18 is writing was fine to the eyes of LCOV from back then but no longer is. That's my current best guess: maybe the LCOV output no longer matches today's LCOV expectations for its input. What do you think?

mozilla / grcov