lcov results differ from CLI?

[gitblit]

What version of Bun is running?

1.1.16+bf7b327f6

What platform is your computer?

Linux 5.15.133.1-microsoft-standard-WSL2 x86_64 x86_64

What steps can reproduce the bug?

This is a challenge. I don't know enough about the lcov format to know if my expectations are incorrect and it's a limitation of the format. Having said that:

bun test --coverage # output
bun test --coverage --coverage-reporter=lcov # output

What is the expected behavior?

The results reported/displayed in the CLI matches the reports rendered by lcov formatters.

What do you see instead?

• ✅ Function reporting % matches between CLI and lcov formatter(s).
• :x: Line reporting differs. CLI shows higher coverage than lcov formatter(s). It appears the LH (lines hit?) undercounts... maybe it's skipping braces and comments and the CLI reporter includes these?

Additional information

Focus on DairyCowRepository.ts:

I'm using livewing/lcov-job-summary@v1.1.0 to render the following results.

TN:
SF:src/DairyCowRepository.ts
FNF:16
FNH:15
DA:1,29
DA:4,20
DA:5,17
DA:6,24
DA:7,31
DA:8,21
DA:9,12
DA:10,19
DA:13,26
DA:14,72
DA:16,17
DA:18,1
DA:20,38
DA:21,87
DA:22,1
DA:24,26
DA:25,63
DA:26,1
DA:28,20
DA:30,1
DA:32,42
DA:35,32
DA:36,115
DA:39,12
DA:40,24
DA:41,25
DA:52,8
DA:54,80
DA:55,24
DA:57,3
DA:60,94
DA:63,20
DA:64,17
DA:65,24
DA:66,31
DA:67,21
DA:68,15
DA:69,19
DA:72,16
DA:73,25
DA:74,24
DA:75,55
DA:76,44
DA:77,20
DA:79,35
DA:88,9
DA:89,33
DA:90,44
DA:91,22
DA:94,66
DA:95,45
DA:96,66
DA:97,1
DA:99,83
LF:101
LH:54
end_of_record

[gitblit]

@Electroid any insight on which side is counting incorrectly? lcov or the CLI reporter?

[gitblit]

In the latest version (1.1.27) which has LCOV fixes in #13633 , the results look a little better. Uncovered lines more closely track the CLI, but there is still major discrepancy on coverage. Is the CLI being too optimistic? Or is the LCOV reporter being pessimistic?

[gitblit]

@fmorency I am curious on the example from your recent fix in #13633, does the Bun CLI and the output from genhtml match?

[fmorency]

@gitblit They didn't match exactly. The fix in #13633 only addresses the inaccurate reporting of uncovered lines in the lcov reporter.