[BUG] Running tests with coverage outputs coverage for auto-generated files in `obj` folder

[glektarssza]

Describe the Issue

Running unit tests inside VSCode with coverage enabled generates coverage metrics for files that are automatically generated by the dotnet SDK and placed inside of the obj folder. For example, LibraryImports.g.cs which is created when using the System.Runtime.InteropServices.LibraryImport attribute to access native code.

As this file does not technically exist on disk it leads to strange behavior (0% coverage, errors when trying to open the file to view, etc.)

See the below screenshot for an example of this issue in action.

Screenshot is from this repository and commit.

Steps To Reproduce

1. Checkout this repository and commit.
2. Run the unit tests with code coverage.

Expected Behavior

Code coverage is not reported for LibraryImports.g.cs or any other files inside of the obj or bin folders.

Environment Information

• OS: Windows 11
• OS Version: 23H3 Build 22631.4169
• VSCode Version: 1.94.0-insider (user setup)
• VSCode Commit: 4f485cf59847506bc1ba2aaab127d31dcbe2c9dc
• Extension Version: v1.11.6 (pre-release)
• ms-dotnettools.csharp Version: v2.46.32 (pre-release)

[AbhitejJohn]

Tagging @fhnaseer to help.

[fhnaseer]

@glektarssza As you mentioned, LibraryImport.g.cs is added because you are using System.Runtime.InteropServices.LibraryImport. We do not exclude .g.cs files by default. Some people want to have coverage for this. You can exclude auto-generated files by adding the following snippet in the runsettings file.

  <Sources>
      <Exclude>
        <Source>.*\.g\.cs$</Source>
      </Exclude>
    </Sources>

https://learn.microsoft.com/en-us/visualstudio/test/customizing-code-coverage-analysis?view=vs-2022#run-settings-file

[glektarssza]

Will test and report back.

[glektarssza]

Side note, apparently the setting dotnet.unitTests.runSettingsPath does not respect VSCode variables.

Setting it to:

"dotnet.unitTests.runSettingsPath": "${workspaceFolder}/coverage.runsettings"

produced the following error:

Cannot read runsettings file at: c:\Users\GlekLaptop\repositories\github\glektarssza\glekcraft\${workspaceFolder}\coverage.runsettings

When I set it to a custom runsettings file, though, I get absolutely spammed with coverage values from the dotnet SDK...

Or at least I assume these are from the dotnet SDK. They might be from the MSTest framework? I'm not sure but that seems more likely.

[fhnaseer]

If no runsettings are provided then we exclude a lot of stuff from the coverage report. When you use a custom runsettings file, then these are not excluded. By looking at your report, I would suggest to add the following in your runsettings file.

        <!-- Match the path of the source files in which each method is defined: -->
        <Sources>
          <Exclude>
            <Source>.*\\atlmfc\\.*</Source>
            <Source>.*\\vctools\\.*</Source>
            <Source>.*\\public\\sdk\\.*</Source>
            <Source>.*\\microsoft sdks\\.*</Source>
            <Source>.*\\vc\\include\\.*</Source>
          </Exclude>
        </Sources>

        <!-- Match the company name property in the assembly: -->
        <CompanyNames>
          <Exclude>
            <CompanyName>.*microsoft.*</CompanyName>
          </Exclude>
        </CompanyNames>

[fhnaseer]

Side note, apparently the setting dotnet.unitTests.runSettingsPath does not respect VSCode variables.

Setting it to:

"dotnet.unitTests.runSettingsPath": "${workspaceFolder}/coverage.runsettings"

produced the following error:

Cannot read runsettings file at: c:\Users\GlekLaptop\repositories\github\glektarssza\glekcraft\${workspaceFolder}\coverage.runsettings

@AbhitejJohn can you create a bug for this and assign it to someone?

[glektarssza]

If no runsettings are provided then we exclude a lot of stuff from the coverage report. When you use a custom runsettings file, then these are not excluded. By looking at your report, I would suggest to add the following in your runsettings file. [SNIPPED]

This has worked, with a bit more added to remove other dependencies I've included in my project. Not exactly intuitive but it has worked. Thanks for the assistance.