2024/01/coverage-reports/

[utterances-bot]

Code Coverage Reports for .NET Projects | Know Your Toolset

Generating local code coverage reports, and getting pipelines set up to evaluate coverage.

https://knowyourtoolset.com/2024/01/coverage-reports/

[ashishbhatt-afk]

Hello, I need some help regarding this. I will try to explain it as much as possible. I want to exclude few files from code coverage but I am unable to do so via .runsettings file. I have tried numerous things. Below is my current YAML and .runsettings. YAML: I have tried a combination of these tasks.

  - task: DotNetCoreCLI@2
    displayName: 'Run Tests and Collect Coverage'
    inputs:
      command: 'test'
      projects: '**/*Tests.csproj'
      arguments: '--configuration $(BUILD_CONFIGURATION) --no-build --settings demo.runsettings'

  # - task: DotNetCoreCLI@2
  #   displayName: 'Run Tests and Collect Coverage'
  #   inputs:
  #     command: 'test'
  #     projects: '**/*Tests.csproj'
  #     arguments: '--configuration $(BUILD_CONFIGURATION) --no-build /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(COVERAGE_DIRECTORY)/ --settings "demo.runsettings"'
  #     publishTestResults: true

  - script: |
      dotnet tool install --global dotnet-reportgenerator-globaltool
      reportgenerator -reports:$(COVERAGE_DIRECTORY)/coverage.cobertura.xml -targetdir:$(COVERAGE_REPORT_DIRECTORY) -reporttypes:Html
    displayName: 'Generate Coverage Report'

  # - task: PublishCodeCoverageResults@2
  #   displayName: 'Publish Code Coverage'
  #   inputs:
  #     summaryFileLocation: '$(COVERAGE_DIRECTORY)/coverage.cobertura.xml'

  - task: PublishCodeCoverageResults@2
    displayName: 'Publish code coverage: Azure DevOps'
    inputs:
      summaryFileLocation: '$(Agent.TempDirectory)/**/*cobertura.xml'

.runsettings file: tried below combination of runsettings.

<?xml version="1.0" encoding="utf-8" ?>

cobertura,opencover .*App\.xaml\.cs **/IRISElementsBookkeeping/App.xaml.cs, **/IRISElementsBookkeeping/App.xaml.cs true

The code coverage still shows this file in the code coverage section of pipeline summary. /Users/runner/work/1/s/IRISElementsBookkeeping/App.xaml.cs

[dahlsailrunner]

The .runsettings file is an XML formatted file -- can you shared the content of that? If you're using the above, that's plain text and won't be recognized as a .runsettings file.

[ashishbhatt-afk]

<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="XPlat code coverage">
        <Configuration>
          <Format>cobertura,opencover</Format>
          <ExcludeByFile>.*App\.xaml\.cs</ExcludeByFile>
          <ExcludeByFile>**/IRISElementsBookkeeping/App.xaml.cs,</ExcludeByFile> 
          <ExcludeByFile>**/IRISElementsBookkeeping/App.xaml.cs</ExcludeByFile> 
          <SkipAutoProps>true</SkipAutoProps>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
</RunSettings>

[dahlsailrunner]

You can only have a single ExcludeByFile node - that may be part of the issue. Here are the formal docs for the .runsettings content: https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/VSTestIntegration.md#advanced-options-supported-via-runsettings

[dahlsailrunner]

and here's a sample file that I've used that works correctly:

<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="XPlat code coverage">
        <Configuration>
          <Format>cobertura,opencover</Format>
          <Exclude>[*.Tests?]*</Exclude> <!-- [Assembly-Filter]Type-Filter -->
          <ExcludeByFile>**/Migrations/*.cs,</ExcludeByFile> <!-- Globbing filter -->
          <SkipAutoProps>true</SkipAutoProps>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
</RunSettings>

[dahlsailrunner]

Another option at your disposal is an attribute inside your code: [ExcludeFromCodeCoverage] can be applied to classes or methods.

[ashishbhatt-afk]

I will give it a try and post here, irrespective of the result. Pass or fail. I appreciate the help. :)

[ashishbhatt-afk]

<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="XPlat code coverage">
        <Configuration>
          <Format>cobertura,opencover</Format>
          <Exclude>[*.Tests?]*</Exclude>
          <ExcludeByFile>**/IRISElementsBookkeeping/*.cs,</ExcludeByFile>
          <SkipAutoProps>true</SkipAutoProps>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
</RunSettings>

I tried with above .runsettings to see if it would work. but still no luck. in my root directory, I have folder IRISElementsBookkeeping that has the .cs files. I want to exclude those.

Below is my YAML:

- task: DotNetCoreCLI@2
        displayName: Build Unit tests
        inputs:
          command: 'build'
          arguments: '--configuration $(BUILD_CONFIGURATION)'
          projects: '$(TEST_PROJECT_PATH)'

      - task: Bash@3
        inputs:
          targetType: 'inline'
          script: |
            pwd
            ls
            brew install findutils
            gfind . -regex '.*App.xaml.cs'

      - task: DotNetCoreCLI@2
        displayName: 'Run Tests and Collect Coverage'
        inputs:
          command: 'test'
          projects: '**/*Tests.csproj'
          arguments: '--configuration $(BUILD_CONFIGURATION) --no-build /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(COVERAGE_DIRECTORY)/ --settings demo.runsettings'
          publishTestResults: true

      - script: |
          dotnet tool install --global dotnet-reportgenerator-globaltool
          reportgenerator -reports:$(COVERAGE_DIRECTORY)/coverage.cobertura.xml -targetdir:$(COVERAGE_REPORT_DIRECTORY) -reporttypes:Html
        displayName: 'Generate Coverage Report'

      - task: PublishCodeCoverageResults@2
        displayName: 'Publish Code Coverage'
        inputs:
          summaryFileLocation: '$(COVERAGE_DIRECTORY)/coverage.cobertura.xml'

[dahlsailrunner]

last questions are probably along the lines of where are your code files relative to the .runsettings file and also are you sure that the .runsettings file is getting picked up at all? you should be able to run the dotnet test command from the command line (on your own workstation) and get the results you're looking for. You can even run the report generator locally too.

[ashishbhatt-afk]

According to https://learn.microsoft.com/en-us/visualstudio/test/customizing-code-coverage-analysis?view=vs-2022#regular-expressions .runsettings file uses regular expressions. In my Yaml if you see I tried using gfind . -regex '.*App.xaml.cs' to see if it actually located this file. and here it actually locates the file.

1. I am sure that the runsettings file is getting picked up.
2. .runsettings file is in the root folder and code files are also with the folder at the root.

It is a bit challenging for me test this locally since pipeline uses a mac-os and i do not have a mac handy.

[dahlsailrunner]

Not sure I have much else to help you with -- running locally would be a good thing (mac / win shouldn't matter for this), but failing that I'd post an issue / question in the https://github.com/coverlet-coverage/coverlet/ repo mentioned above.