microsoft / vstest

Visual Studio Test Platform is the runner and engine that powers test explorer and vstest.console.
MIT License
869 stars 316 forks source link

InProcess coverlet data collector isn't running when enabling coverlet from the runsettings file #2432

Open ViktorHofer opened 4 years ago

ViktorHofer commented 4 years ago

If the coverlet data collector is configured via a .runsettings file and set to enabled=True, the InProcessDataCollector isn't added to the vstest.console's internal .runsettings memory representation. The impact is that the code coverage measurement runs as expected but the results aren't stable because of the missing in-proc collector: https://github.com/coverlet-coverage/coverlet/issues/834.

Example runsettings file in which coverlet data collector is enabled:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="XPlat code coverage" enabled="True">
        <Configuration>
          <Include>[System.Text.RegularExpressions]*</Include>
          <ExcludeByFile>C:\git\runtime3\src\libraries\Common\src\System\SR.*,C:\git\runtime3\src\libraries\Common\src\System\NotImplemented.cs</ExcludeByFile>
          <IncludeDirectory>C:\git\runtime3\artifacts\bin\testhost\net5.0-Windows_NT-Debug-x64\shared\Microsoft.NETCore.App\5.0.0</IncludeDirectory>
          <Format>opencover</Format>
          <SingleHit>false</SingleHit>
          <UseSourceLink>true</UseSourceLink>
          <IncludeTestAssembly>false</IncludeTestAssembly>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
</RunSettings>

I have a fix for that locally and will submit a PR.

cc @MarcoRossignoli

Evangelink commented 1 year ago

@MarcoRossignoli Could you please investigate this when you have some time?