dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.05k stars 4.69k forks source link

Code coverage fails on some assemblies #103190

Open stephentoub opened 3 months ago

stephentoub commented 3 months ago

Coverage is working fine on some assemblies, e.g. dotnet build /t:test /p:Coverage on channels:

    Discovering: System.Threading.Channels.Tests (method display = ClassAndMethod, method display options = None)
    Discovered:  System.Threading.Channels.Tests (found 538 test cases)
    Starting:    System.Threading.Channels.Tests (parallel test collections = on [12 threads], stop on fail = off)
      System.Threading.Channels.Tests.StressTests.CanceledReads [SKIP]
        Condition(s) not met: "IsStressModeEnabled"
      System.Threading.Channels.Tests.StressTests.ReadWriteVariations [SKIP]
        Condition(s) not met: "IsStressModeEnabled"
    Finished:    System.Threading.Channels.Tests
  === TEST EXECUTION SUMMARY ===
     System.Threading.Channels.Tests  Total: 1362, Errors: 0, Failed: 0, Skipped: 2, Time: 10.065s

  Calculating coverage result...
    Generating report 'D:\repos\runtime\artifacts\bin\System.Threading.Channels.Tests\Debug\net9.0\coverage.opencover.xml'
  +---------------------------+--------+--------+--------+
  | Module                    | Line   | Branch | Method |
  +---------------------------+--------+--------+--------+
  | System.Threading.Channels | 97.74% | 96.7%  | 97.44% |
  +---------------------------+--------+--------+--------+

  +---------+--------+--------+--------+
  |         | Line   | Branch | Method |
  +---------+--------+--------+--------+
  | Total   | 97.74% | 96.7%  | 97.44% |
  +---------+--------+--------+--------+
  | Average | 97.74% | 96.7%  | 97.44% |
  +---------+--------+--------+--------+

But on others, it's failing to produce any data, e.g. on immutable collections:

    Discovering: System.Collections.Immutable.Tests (method display = ClassAndMethod, method display options = None)
    Discovered:  System.Collections.Immutable.Tests (found 4287 of 4392 test cases)
    Starting:    System.Collections.Immutable.Tests (parallel test collections = on [12 threads], stop on fail = off)
    Finished:    System.Collections.Immutable.Tests
  === TEST EXECUTION SUMMARY ===
     System.Collections.Immutable.Tests  Total: 20604, Errors: 0, Failed: 0, Skipped: 0, Time: 5.713s

  Calculating coverage result...
    Generating report 'D:\repos\runtime\artifacts\bin\System.Collections.Immutable.Tests\Debug\net9.0\coverage.opencover.xml'
  +------------------------------+------+--------+--------+
  | Module                       | Line | Branch | Method |
  +------------------------------+------+--------+--------+
  | System.Collections.Immutable | 0%   | 0%     | 0%     |
  +------------------------------+------+--------+--------+

  +---------+------+--------+--------+
  |         | Line | Branch | Method |
  +---------+------+--------+--------+
  | Total   | 0%   | 0%     | 0%     |
  +---------+------+--------+--------+
  | Average | 0%   | 0%     | 0%     |
  +---------+------+--------+--------+
dotnet-policy-service[bot] commented 3 months ago

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries See info in area-owners.md if you want to be subscribed.

ViktorHofer commented 3 months ago

But on others, it's failing to produce any data, e.g. on immutable collections:

Is there any information available in a log on why? Is the assembly instrumented correctly before running? Here are some docs / env var settings on how to get more detailed logs: https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/Troubleshooting.md#enable-injected-tracker-log

stephentoub commented 3 months ago

But on others, it's failing to produce any data, e.g. on immutable collections:

Is there any information available in a log on why? Is the assembly instrumented correctly before running? Here are some docs / env var settings on how to get more detailed logs: https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/Troubleshooting.md#enable-injected-tracker-log

Nothing I saw that suggested what the problem was, but I'd appreciate you trying.