danielpalme / ReportGenerator

ReportGenerator converts coverage reports generated by coverlet, OpenCover, dotCover, Visual Studio, NCover, Cobertura, JaCoCo, Clover, gcov or lcov into human readable reports in various formats.
https://reportgenerator.io
Apache License 2.0
2.56k stars 279 forks source link

Question about lcov conversion #673

Closed ricardoboss closed 3 months ago

ricardoboss commented 3 months ago

Hi there!

I have a question about the conversion from an lcov.info file to cobertura.xml:

Why do uncovered files appear with a line rate of 1?

Let's look a an example:

This is part of my lcov.info:

SF:some/file/in/my/repo.txt
FNH:0
BRH:0
LH:0
LF:21
end_of_record

This can be read as (at least AFAIK):

After the conversion by ReportGenerator, the cobertura reports something slightly different:

(most content omitted for brevity)

        <class name="some/file/in/my/repo.txt" filename="some/file/in/my/repo.txt" line-rate="1" branch-rate="1" complexity="NaN">
          <methods />
          <lines />
        </class>

...which yields a coverage of 100%, even though the lcov file reports no covered lines.

Is this a bug? I would have expected it to report 0% coverage.

danielpalme commented 3 months ago

Mathematically speaking, the result is not defined because division by 0 is not allowed. So NaN would be the best option. But that would cause other tools to fail, when they read the generated Cobertura file.

Since the lcov.info file does not contain any information about covered/uncovered lines, you can argue both ways:

Both statements are correct and incorrect. I decided to handle this as 100% coverage.

ricardoboss commented 3 months ago

I see. Thanks for explaining!

I was confused because only a few files were covered by tests, but coverage showed about 80%. The metric would be more helpful in determining how much code is covered by tests when it would incorporate uncovered lines (as in lines covered / lines in total).

But I guess you can't do anything about the way cobertura and/or lcov are designed. Do you see a way to make this configurable for the reports? Would you even want to support that? What do you think?

danielpalme commented 3 months ago

The problem is that the lcov file does not contain any information on these lines. Then the old rule "Garbage in, garbage out" applies.

I don't want to make that configurable. I consider that as an edge case.
I recommend to use a different input format than lcov, if that's an option for you.

If you really want to change the behavior, you can create a custom Cobertura output generator (see https://github.com/danielpalme/ReportGenerator/wiki/Custom-reports).

You could basically use my implementation and change those two lines:

https://github.com/danielpalme/ReportGenerator/blob/2b62fbcfeabfe75b1c5d0224987f8fcb23adf87b/src/ReportGenerator.Core/Reporting/Builders/CoberturaReportBuilder.cs#L197-L198

ricardoboss commented 3 months ago

I recommend to use a different input format than lcov, if that's an option for you.

Unfortunately not, no... Maybe in the future.

If you really want to change the behavior [...]

Well, it's a luxury problem, don't get me wrong. I am very happy otherwise. Ideally I'd have a test for all files anyways :D

Thanks for your support!