SteveGilham / altcover

Cross-platform coverage gathering and processing tool set for dotnet/.Net Framework and Mono
MIT License
494 stars 17 forks source link

AltCover generates report with integer numbers that exceed Int32.MaxValue #195

Closed denis-gvardionov closed 9 months ago

denis-gvardionov commented 9 months ago

When using AltCover version 8.6.68 for analyzing the code coverage in the project, AltCover generates a coverage report in XML format, where some integer numbers exceed the System.Int32.MaxValue upper limit (2147483647). And because of this the subsequent procedure cannot import such a report, because cannot parse the too big integer value.

The error message is the next:

WARN: Could not import coverage report 'C:......\results.net6.0.xml' because 'Expected an integer instead of "2391726432" for the attribute "vc" in C:.......\results.net6.0.xml at line 39797'

And the next:

WARN: Could not import coverage report 'C:......\results.net6.0.xml' because 'Expected an integer instead of "3541295175" for the attribute "vc" in C:.......\results.net6.0.xml at line 38744'

Here is a part of report, generated by the AltCover, where such invalid value is located:

<Method visited="true" cyclomaticComplexity="2" nPathComplexity="2" sequenceCoverage="100" branchCoverage="100" isConstructor="false" isStatic="false" isGetter="true" isSetter="false" crapScore="2">
  <Summary numSequencePoints="7" visitedSequencePoints="7" numBranchPoints="3" visitedBranchPoints="3" sequenceCoverage="100" branchCoverage="100" maxCyclomaticComplexity="2" minCyclomaticComplexity="2" visitedClasses="0" numClasses="0" visitedMethods="1" numMethods="1" minCrapScore="2" maxCrapScore="2" />
  <MetadataToken>100664696</MetadataToken>
  <Name>System.Int32 GroupDocs.Editor.HtmlCss.Tools.NonEmptyCollection`1::get_Count()</Name>
  <FileRef uid="190" />
  <SequencePoints>
    <SequencePoint vc="2391726432" uspid="18774" ordinal="0" offset="0" sl="134" sc="13" el="134" ec="14" bec="0" bev="0" fileid="190" />
    <SequencePoint vc="2391726432" uspid="18773" ordinal="1" offset="1" sl="135" sc="17" el="135" ec="75" bec="2" bev="2" fileid="190" />
    <SequencePoint vc="2304973709" uspid="18772" ordinal="2" offset="16" sl="136" sc="17" el="136" ec="18" bec="0" bev="0" fileid="190" />
    <SequencePoint vc="2304973709" uspid="18771" ordinal="3" offset="17" sl="137" sc="21" el="137" ec="30" bec="0" bev="0" fileid="190" />
    <SequencePoint vc="86752723" uspid="18770" ordinal="4" offset="21" sl="140" sc="17" el="140" ec="18" bec="0" bev="0" fileid="190" />
    <SequencePoint vc="86752723" uspid="18769" ordinal="5" offset="22" sl="141" sc="21" el="141" ec="55" bec="0" bev="0" fileid="190" />
    <SequencePoint vc="2391726432" uspid="18768" ordinal="6" offset="38" sl="143" sc="13" el="143" ec="14" bec="0" bev="0" fileid="190" />
  </SequencePoints>
  <BranchPoints>
    <BranchPoint vc="2304973709" uspid="6350" ordinal="0" offset="14" sl="135" path="0" offsetend="16" fileid="190" />
    <BranchPoint vc="86752723" uspid="6351" ordinal="1" offset="14" sl="135" path="1" offsetend="21" fileid="190" />
  </BranchPoints>
  <MethodPoint xsi:type="SequencePoint" vc="2391726432" uspid="18774" ordinal="0" offset="0" sl="134" sc="13" el="134" ec="14" bec="0" bev="0" fileid="190" />
</Method>

After subsequent investigations we’ve found that AltCover generates such out-of-range-integers not for the random code, but for the two specific methods, located within two classes. We made a workaround by excluding these two classes from the analysis by using a “FileFilter” property from the “coverage prepare options” class, which implements the “AltCover.Cake.PrepareOptions”. After such exclusion all is working well.

We tried different ways to figure out why the AltCover generates out-of-range-integers especially for specific code pieces, but without luck. There is a very usual code there, nothing special.

Our project is a closed-source commercial library, so we cannot share the whole of its code base. But we can share these two files, where problematic code is located. We also share the reports generated by AltCover.

List of attached files:

7z archive with these 4 files is located here: https://drive.google.com/file/d/18A9C1awXcrXWztWFLG7qMVLygA3qEx7X/view?usp=sharing (I cannot attach files directly because they are too big)

SteveGilham commented 9 months ago

Coverage numbers like this are indeed orders of magnitude higher than I've observed in real life testing in my corporate career; hence no limiter has been built in, as that would incur a cost on every visit, for something expected never to happen.

Setting the prepare options SingleVisit property to true will paper over the problem by replacing visit counts by an effective boolean - 1 or 0 for visited or not.

Inspecting the quoted fragment of the coverage report and the matching source, the values are consistent with 2391726432 times when a collection is being inspected, of which only 3.6% have more than one member. The number of non-trivial collections in get_Count approximately match up with the number of calls to the [] indexer (aka get_Item) that fetch items at non-negative offsets, of which 2.6% are at indexes above 0.

This looks to me like the coverage tool is, as a side-effect, pointing out a hot-spot in the library as implemented. The methods within the NonEmptyCollection type are rather fond of invoking the Count property rather than caching and managing it; this including the enumerator type. This looks like it is ripe for cases where innocent looking application code makes a call that does more repetitive work than would be apparent on the surface.

SteveGilham commented 9 months ago

Out of interest, I made the supplied code fragments buildable and passed my Gendarme tool over it; notifications that the code is stubbed aside, the majority of the reports are of the repeated invocations of ICollection1::get_Count(),IEnumerator1::get_Current() and NonEmptyCollection1::get_Count()`

denis-gvardionov commented 9 months ago

Thanks a lot for the answers and code investigation. I used your advises and modified the NonEmptyCollection in the next way:

As a result, the AltCover report has completed without issues, code coverage is working, and the VisitedCount for the NonEmptyCollection.Count property is 1539339537, while for the CollectionTools.IsNullOrEmpty is 1149568743, both of which are lesser than Int32.MaxValue.

So the issue is completely solved for me, thanks a lot one more time, I'm closing this ticket as "Resolved".