empty code-coverage when coverlet runs on project which refrences the nuget-package

[earloc]

repro steps

See full repro-sample here: https://github.com/earloc/TypealizR/tree/repros/no-code-coverage/repros/30_Empty_CodeCoverage

• create an app

• create an XUnit test-project for the app

• create a test-case hitting something from within app

• add coverlet.msbuild to the test-project

• configure coverlet-properties in the test-project

  <PropertyGroup>
      <CollectCoverage>true</CollectCoverage>
      <ExcludeByAttribute>GeneratedCodeAttribute</ExcludeByAttribute>
      <CoverletOutputFormat>lcov,opencover</CoverletOutputFormat>
  </PropertyGroup>

• run dotnet test on the test-projet

• reproduce the issue

  • ReproStep.A: add TypealizR to the app
  • ReproStep.B: configure ResX-files to be visible to analyzers

    <PropertyGroup>
    <!-- Update the property to include all EmbeddedResource files -->
    <AdditionalFileItemNames>$(AdditionalFileItemNames);EmbeddedResource</AdditionalFileItemNames>
    </PropertyGroup>

expected

an output like this:

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: < 1 ms - XUnit.Tests.dll (net6.0)

Calculating coverage result...
  Generating report '.../TypealizR\repros\30_Empty_CodeCoverage\XUnit.Tests\coverage.info'
  Generating report '.../TypealizR\repros\30_Empty_CodeCoverage\XUnit.Tests\coverage.opencover.xml'

+------------+------+--------+--------+
| Module     | Line | Branch | Method |
+------------+------+--------+--------+
| ConsoleApp | 60%  | 100%   | 66.66% |
+------------+------+--------+--------+

+---------+------+--------+--------+
|         | Line | Branch | Method |
+---------+------+--------+--------+
| Total   | 60%  | 100%   | 66.66% |
+---------+------+--------+--------+
| Average | 60%  | 100%   | 66.66% |
+---------+------+--------+--------+

generated coverage-files contain detailed coverage-data

actual

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: < 1 ms - XUnit.Tests.dll (net6.0)

Calculating coverage result...
  Generating report '...\TypealizR\repros\30_Empty_CodeCoverage\XUnit.Tests\coverage.info'
  Generating report '...\TypealizR\repros\30_Empty_CodeCoverage\XUnit.Tests\coverage.opencover.xml'

+--------+------+--------+--------+
| Module | Line | Branch | Method |
+--------+------+--------+--------+

+---------+------+--------+--------+
|         | Line | Branch | Method |
+---------+------+--------+--------+
| Total   | 0%   | 0%     | 0%     |
+---------+------+--------+--------+
| Average | 0%   | 0%     | 0%     |
+---------+------+--------+--------+

generated coverage-files are (kind-of) empty

[earloc]

Problem does not occur when just doing ReproStp.A. If we leave out ReproStep.B, everything works as expected.

If leaving out ReproStep. B, there are no files generated, hence it looks like coverage is working again.

[earloc]

looks like coverlet is unable to instrument the generated files:

Target "InstrumentModules" in file "...\.nuget\packages\coverlet.msbuild\3.2.0\build\coverlet.msbuild.targets" from project "...\repros\30_Empty_CodeCoverage\XUnit.Tests\XUnit.Tests.csproj" (target "InstrumentModulesAfterBuild" depends on it):
       Using "Coverlet.MSbuild.Tasks.InstrumentationTask" task from assembly "...\.nuget\packages\coverletTask "Coverlet.MSbuild.Tasks.InstrumentationTask"
         [coverlet] Unable to instrument module: ...\repros\30_Empty_CodeCoverage\XUnit.Tests\bin\Debug\net6.0\ConsoleApp.dll, pdb without local source files, [...\repros\30_Empty_CodeCoverage\ConsoleApp\TypealizR.SourceGenerators\TypealizR.SourceGenerators.StringLocalizer.SourceGenerator\IStringLocalizerExtensions_ConsoleApp.Resource1.cs]
         [coverlet] Unable to instrument module: ...\repros\30_Empty_CodeCoverage\XUnit.Tests\bin\Debug\net6.0\xunit.assert.dll, embedded pdb without local source files, [C:\Dev\xunit\xunit\src\xunit.assert\Asserts\Assert.cs]
         [coverlet] Unable to instrument module: ...\repros\30_Empty_CodeCoverage\XUnit.Tests\bin\Debug\net6.0\xunit.core.dll, embedded pdb without local source files, [C:\Dev\xunit\xunit\src\common\ExceptionExtensions.cs]
         [coverlet] Unable to instrument module: ...\repros\30_Empty_CodeCoverage\XUnit.Tests\bin\Debug\net6.0\xunit.execution.dotnet.dll, embedded pdb without local source files, [C:\Dev\xunit\xunit\src\xunit.execution\Extensions\ReflectionAbstractionExtensions.cs]
         [coverlet] Unable to instrument module: ...\repros\30_Empty_CodeCoverage\XUnit.Tests\bin\Debug\net6.0\xunit.runner.reporters.netcoreapp10.dll, embedded pdb without local source files, [C:\Dev\xunit\xunit\src\xunit.runner.reporters\Utility\EnvironmentHelper.cs]
         [coverlet] Unable to instrument module: ...\repros\30_Empty_CodeCoverage\XUnit.Tests\bin\Debug\net6.0\xunit.runner.utility.netcoreapp10.dll, embedded pdb without local source files, [C:\Dev\xunit\xunit\src\xunit.runner.utility\Extensions\DisposableExtensions.cs]
         [coverlet] Unable to instrument module: ...\repros\30_Empty_CodeCoverage\XUnit.Tests\bin\Debug\net6.0\xunit.runner.visualstudio.dotnetcore.testadapter.dll, embedded pdb without local source files, [/_/src/xunit.runner.visualstudio/obj/Release/netcoreapp2.1/xunit.runner.visualstudio.dotnetcore.testadapter.Version.cs]
       Done executing task "Coverlet.MSbuild.Tasks.InstrumentationTask".

[earloc]

maybe connected to https://github.com/coverlet-coverage/coverlet/issues/1378

https://github.com/coverlet-coverage/coverlet/issues/1377

[earloc]

possible workaround

fall back to using coverlet 2.5.1

<PackageReference Include="coverlet.msbuild" Version="2.5.1">
  <PrivateAssets>all</PrivateAssets>
  <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>

starting from coverlet 2.6.0 seems to provoke the issue:

<PackageReference Include="coverlet.msbuild" Version="2.6.0">
  <PrivateAssets>all</PrivateAssets>
  <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>

[earloc]

The generated sources itself do not seem to be the problem. When emitting the generated files to the file-system, move them into the project hierarchy and removing the source-generator, coverage comes out as intended. Sadly, this is not a viable workaround 😉.

[earloc]

The generated sources itself does not be the problem. When emitting the generated files to the file-system, move them into the project hierarchy and removing the source-generator, coverage comes out as intended. Sadly, this is not a viable workaround 😉.

[earloc]

Trying out solution suggested in https://github.com/coverlet-coverage/coverlet/issues/1322

[earloc]

confirmed: appending "g.cs" fixes the issue