danielpalme / ReportGenerator

ReportGenerator converts coverage reports generated by coverlet, OpenCover, dotCover, Visual Studio, NCover, Cobertura, JaCoCo, Clover, gcov or lcov into human readable reports in various formats.
https://reportgenerator.io
Apache License 2.0
2.56k stars 279 forks source link

Visual Studio Coverage: Support partial line coverage #629

Closed RoadTrain closed 9 months ago

RoadTrain commented 10 months ago

Describe the bug VS Coverage (.coveragexml) supports partial line coverage, as does ReportGenerator, but it doesn't correctly parse partially covered lines.

Line coverage is represented by blocks like this:

<Lines>
    <LnStart>31</LnStart>
    <ColStart>17</ColStart>
    <LnEnd>31</LnEnd>
    <ColEnd>40</ColEnd>
    <Coverage>0</Coverage>
    <SourceFileID>6673</SourceFileID>
    <LineID>268435</LineID>
</Lines>

Coverage element can have the following values (couldn't find any doc, had to deduct it from coverage):

0 - Covered
1 - Partially Covered
2 - Not Covered

Case 1: Line is partially covered

A simple case when the entire line is just covered partially.

<Method>
    <MethodKeyName>myAssembly.dll786628a7-db23-4a62-8430-5c6354d84c0fNamespaceMyClass!GetHashCode()!11233</MethodKeyName>
    <MethodName>GetHashCode()</MethodName>
    <MethodFullName>GetHashCode()</MethodFullName>
    <LinesCovered>0</LinesCovered>
    <LinesPartiallyCovered>1</LinesPartiallyCovered>
    <LinesNotCovered>0</LinesNotCovered>
    <BlocksCovered>4</BlocksCovered>
    <BlocksNotCovered>1</BlocksNotCovered>
    <Lines>
        <LnStart>64</LnStart>
        <ColStart>46</ColStart>
        <LnEnd>64</LnEnd>
        <ColEnd>74</ColEnd>
        <Coverage>1</Coverage>
        <SourceFileID>6673</SourceFileID>
        <LineID>267487</LineID>
    </Lines>
</Method>

Case 2: Line is partially covered (part of line is covered, part is not)

Sometimes there's a more detailed data on which part of the line is covered, and which isn't. The whole line is considered partially covered. In this example, line 31 is covered partially, which is reflected in method metrics.

<Method>
    <MethodKeyName>myAssembly.dll786628a7-db23-4a62-8430-5c6354d84c0fNamespaceMyClass!set_Wildcard(string)!11644</MethodKeyName>
    <MethodName>set_Wildcard(string)</MethodName>
    <MethodFullName>set_Wildcard(string)</MethodFullName>
    <LinesCovered>4</LinesCovered>
    <LinesPartiallyCovered>1</LinesPartiallyCovered>
    <LinesNotCovered>0</LinesNotCovered>
    <BlocksCovered>5</BlocksCovered>
    <BlocksNotCovered>1</BlocksNotCovered>
    <Lines>
        <LnStart>31</LnStart>
        <ColStart>17</ColStart>
        <LnEnd>31</LnEnd>
        <ColEnd>40</ColEnd>
        <Coverage>0</Coverage>
        <SourceFileID>6673</SourceFileID>
        <LineID>268435</LineID>
    </Lines>
    <Lines>
        <LnStart>31</LnStart>
        <ColStart>41</ColStart>
        <LnEnd>31</LnEnd>
        <ColEnd>48</ColEnd>
        <Coverage>2</Coverage>
        <SourceFileID>6673</SourceFileID>
        <LineID>268436</LineID>
    </Lines>
    <Lines>
        <LnStart>32</LnStart>
        <ColStart>17</ColStart>
        <LnEnd>32</LnEnd>
        <ColEnd>35</ColEnd>
        <Coverage>0</Coverage>
        <SourceFileID>6673</SourceFileID>
        <LineID>268437</LineID>
    </Lines>
    <Lines>
        <LnStart>33</LnStart>
        <ColStart>17</ColStart>
        <LnEnd>33</LnEnd>
        <ColEnd>56</ColEnd>
        <Coverage>0</Coverage>
        <SourceFileID>6673</SourceFileID>
        <LineID>268438</LineID>
    </Lines>
    <Lines>
        <LnStart>34</LnStart>
        <ColStart>17</ColStart>
        <LnEnd>34</LnEnd>
        <ColEnd>74</ColEnd>
        <Coverage>0</Coverage>
        <SourceFileID>6673</SourceFileID>
        <LineID>268439</LineID>
    </Lines>
    <Lines>
        <LnStart>35</LnStart>
        <ColStart>13</ColStart>
        <LnEnd>35</LnEnd>
        <ColEnd>14</ColEnd>
        <Coverage>0</Coverage>
        <SourceFileID>6673</SourceFileID>
        <LineID>268440</LineID>
    </Lines>
</Method>

I think this can be mapped to ReportGenerator's LineVisitStatus.PartiallyCovered here: https://github.com/danielpalme/ReportGenerator/blob/3b854bc406e72410f1c80b7bdccd86d3b3b056f5/src/ReportGenerator.Core/Parser/VisualStudioParser.cs#L221-L223

danielpalme commented 10 months ago

You are right. I fixed that in fc1de1c8dd4ecc9a021d13d02fed6c299161c867.

The next release will contain the change. I plan the next release for November, once .NET8 is available.

RoadTrain commented 10 months ago

What about Case 2? When a single line is both covered and not covered, it should be considered partially covered (since ReportGenerator doesn't support coverage mapping by lines and columns). I don't see that in a commit, but maybe I'm wrong :)

danielpalme commented 10 months ago

What about Case 2? When a single line is both covered and not covered, it should be considered partially covered (since ReportGenerator doesn't support coverage mapping by lines and columns). I don't see that in a commit, but maybe I'm wrong :)

No, you are not wrong. In your example line 31 would be considered as covered. I will see, if I can find a good solution for that case. It might easily become complex, if line elements are overlapping (I don't know if that can happen in practice).

RoadTrain commented 10 months ago

What about Case 2? When a single line is both covered and not covered, it should be considered partially covered (since ReportGenerator doesn't support coverage mapping by lines and columns). I don't see that in a commit, but maybe I'm wrong :)

No, you are not wrong. In your example line 31 would be considered as covered. I will see, if I can find a good solution for that case. It might easily become complex, if line elements are overlapping (I don't know if that can happen in practice).

Thanks. With the assumption that the line elements for a particular method cannot overlap it seems doable with something like a two-dimensional array LineVisitStatus[][] where each line is mapped to all statuses within that line.

If needed, we can test it on our fairly large report to see if anything goes wrong.

danielpalme commented 10 months ago

Just made the necessary changes in 8278973bdbe73dbdb9f59b00f4ac23afde0c4d5e. I will let you know once the next release is available.

danielpalme commented 9 months ago

Release 5.2.0 is now available.