Updated version not collecting code coverage

[MaxChal]

Description

Since the latest release our xunit code coverage is no longer being displayed in Sonar Cloud and we fail the new code coverage rule. The only changes we can see in our pipeline was during the prepare phase.

Currently it is showing

SonarScanner for MSBuild 5.0 Using the .NET Core version of the Scanner for MSBuild

Previously it was:

SonarScanner for MSBuild 4.10 Using the .NET Framework version of the Scanner for MSBuild

Pipeline Snippet for our Asp.Net MVC app (.Net Framework)

  - task: NuGetToolInstaller@0
        displayName: "Install NuGet"
        inputs:
          checkLatest: true
          versionSpec: ">4.8.0"

      - task: NuGetCommand@2
        displayName: "Restore Nuget"
        inputs:
          restoreSolution: "**/*.sln"
          feedsToUse: select
          vstsFeed: <>

      - task: SonarCloudPrepare@1
        displayName: "Sonar Cloud Prepare"
        inputs:
          SonarCloud: "<>"
          organization: "<>"
          scannerMode: "MSBuild"
          projectKey: "<>"
          projectName: "<>"
          projectVersion: $(version.full)
          extraProperties: sonar.exclusions=**/AssemblyInfo.cs

      - task: VSBuild@1
        displayName: "Build solution "
        inputs:
          solution: "**/*.sln"
          vsVersion: "latest"
          msbuildArgs: "/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:AutoParameterizationWebConfigConnectionStrings=False"
          platform: "$(BuildPlatform)"
          configuration: "$(BuildConfiguration)"
          clean: true

      - task: VSTest@2
        displayName: "Run unit tests"
        inputs:
          testSelector: "testAssemblies"
          testAssemblyVer2: "**\\*test*.dll\n!**\\obj\\**"
          searchFolder: "$(System.DefaultWorkingDirectory)\\src\\"
          runOnlyImpactedTests: false
          runInParallel: true
          runTestsInIsolation: false
          codeCoverageEnabled: true
          otherConsoleOptions: "/Framework:framework45"
          dontDistribute: false
          platform: "$(BuildPlatform)"
          configuration: "$(BuildConfiguration)"
          diagnosticsEnabled: True

      - task: PublishSymbols@1
        displayName: "Publish symbols path"

      - task: SonarCloudAnalyze@1
        displayName: "Sonar Cloud Analyze"

      - task: SonarCloudPublish@1
        displayName: Sonar Cloud Publish

[amilochau]

Hello,

We encounter the same problem on our Azure DevOps pipeline, since the new version of SonarScanner, for both .NET Framework and .NET Core projects:

• Code Coverage dropped to 0.0%
• Lines to Cover significantly changed

.NET Framework projects

Currently it is showing:

SonarScanner for MSBuild 5.0 Using the .NET Core version of the Scanner for MSBuild

Previously it was:

SonarScanner for MSBuild 4.10 Using the .NET Framework version of the Scanner for MSBuild

Pipeline snippet is:

    steps:
    - task: NuGetToolInstaller@1
      displayName: 'Use NuGet 5.4.0'
      inputs:
        versionSpec: 5.4.0

    - task: NuGetCommand@2
      displayName: NuGet restore
      inputs:
        command: 'restore'
        restoreSolution: '$(restoreBuildSolution)'
        feedsToUse: 'config'
        nugetConfigPath: 'NuGet.config'

    - task: SonarCloudPrepare@1
      displayName: 'Prepare analysis on SonarCloud'
      inputs:
        SonarCloud: 'XXX'
        organization: 'XXX'
        scannerMode: 'MSBuild'
        projectKey: '$(sonarProjectKey)'
        projectName: '$(sonarProjectKey)'
        extraProperties: '$(sonarExtraProperties)'

    - task: VSBuild@1
      displayName: 'Build solution'
      inputs:
        solution: '$(restoreBuildSolution)'
        vsVersion: 16.0
        platform: '$(buildPlatform)'
        configuration: '$(buildConfiguration)'

    - task: VSTest@2
      displayName: 'Run tests and coverage'
      inputs:
        testAssemblyVer2: '$(testProjects)'
        codeCoverageEnabled: true
        platform: '$(buildPlatform)'
        configuration: '$(buildConfiguration)'

    - task: SonarCloudAnalyze@1
      displayName: 'Run Code Analysis'

    - task: SonarCloudPublish@1
      displayName: 'Publish Quality Gate Result'
      inputs:
        pollingTimeoutSec: '300'

.NET Core projects

Currently it is showing:

SonarScanner for MSBuild 5.0 Using the .NET Core version of the Scanner for MSBuild

Previously it was:

SonarScanner for MSBuild 4.10 Using the .NET Framework version of the Scanner for MSBuild

Pipeline snippet is:

    steps:
    - task: DotNetCoreCLI@2
      displayName: Restore
      inputs:
        command: restore
        projects: '$(restoreBuildProjects)'
        feedsToUse: 'config'
        nugetConfigPath: 'nuget.config'

    - task: SonarCloudPrepare@1
      displayName: 'Prepare analysis on SonarCloud'
      inputs:
        SonarCloud: 'XXX'
        organization: 'XXX'
        scannerMode: 'MSBuild'
        projectKey: '$(sonarProjectKey)'
        projectName: '$(sonarProjectKey)'

    - task: DotNetCoreCLI@2
      displayName: Build
      inputs:
        projects: '$(restoreBuildProjects)'
        arguments: '--configuration $(buildConfiguration)'

    - task: DotNetCoreCLI@2
      displayName: 'Test (Unit Tests)'
      inputs:
        command: test
        projects: '$(testProjects)'
        arguments: '--configuration $(buildConfiguration) --collect "Code coverage"'

    - task: SonarCloudAnalyze@1
      displayName: 'Run Code Analysis'

    - task: SonarCloudPublish@1
      displayName: 'Publish Quality Gate Result'
      inputs:
        pollingTimeoutSec: '300'

This problem broke all our pipelines, used by dozens of devs for daily work 😪 Thanks in advance for your help!

[BugsOli]

Hey guys, I've been using this YAML template for code coverage publishing and it works well, I also needed to fix the problem with the new version of the SonarQubePrepare that broke my build (you need version 4.11.0 for C# code it seems):

stages:
- stage: Build
  jobs:
  - job: NETCore
    pool:
      name: Hosted VS2017
    steps:
    - task: SonarQubePrepare@4.11.0
      displayName: 'Sonar - Prepare Analysis'
      inputs:
        SonarQube: 'YOUR_SERVICE_CONNECTION_NAME'
        projectKey: 'YOUR_KEY'
        projectName: 'YOUR_PROJECT_NAME'
        extraProperties: 'sonar.coverageReportPaths=$(Build.SourcesDirectory)/coverlet/reports/SonarQube.xml'

    - task: DotNetCoreCLI@2
      displayName: 'Solution - Build'
      inputs:
        command: build
        projects: '$(Solution)'
        arguments: '--configuration $(Build.Configuration)'

    - task: DotNetCoreCLI@2
      displayName: 'Tests - Execute'
      inputs:
        command: test
        projects: '**/*.Tests.csproj'
        arguments: -c $(Build.Configuration) --collect:"XPlat Code Coverage" -- RunConfiguration.DisableAppDomain=true
        testRunTitle: 'Run Test and collect Coverage'
      continueOnError: false

    - task: DotNetCoreCLI@2
      displayName: 'Tools - Install ReportGenerator'
      inputs:
        command: custom
        custom: tool
        arguments: install --tool-path . dotnet-reportgenerator-globaltool

    - script: reportgenerator -reports:$(Agent.TempDirectory)/**/coverage.cobertura.xml -targetdir:$(Build.SourcesDirectory)/coverlet/reports -reporttypes:"HtmlInline_AzurePipelines;Cobertura;SonarQube"
      displayName: 'Tests - Create Reports'

    - task: PublishCodeCoverageResults@1
      displayName: 'Tests - Publish Coverage'
      inputs:
        codeCoverageTool: Cobertura
        summaryFileLocation: $(Build.SourcesDirectory)/coverlet/reports/Cobertura.xml

    - task: SonarQubeAnalyze@4
      displayName: 'Sonar - Analyze'

    - task: SonarQubePublish@4
      displayName: 'Sonar - Publish Analysis Results'
      inputs:
        pollingTimeoutSec: '300'

    - task: DotNetCoreCLI@2
      displayName: 'Solution - Publish'
      inputs:
        command: publish
        publishWebProjects: True
        arguments: '--configuration $(Build.Configuration) --output $(Build.ArtifactStagingDirectory)'
        zipAfterPublish: True

    - task: PublishPipelineArtifact@1
      displayName: 'Artifact - Publish'
      inputs:
        targetPath: '$(Build.ArtifactStagingDirectory)'
        artifactName: YOUR_ARTIFACT_NAME

I'm using the Report Generator tool to create code coverage reports in various format: one for DevOps and one for SonarQube

[raherrebrugh]

Issue Sonar no longer recognizes any code coverage.

Observations

1. SonarScanner for MSBuild has been upgraded from 4.1 to 5.0
2. TRX files are generated but no longer converted to .coveragexml (with 4.1 the following log entries are found, but missing in 5.0)

   INFO: Parsing the Visual Studio coverage XML report D:\a\_temp\37a1e54f-60bb-4d03-8364-f09756a14729\VssAdministrator_WIN-TOJS93SFU8U_2020-11-03.22_21_21.coveragexml
   INFO: Adding this code coverage report to the cache for later reuse: D:\a\_temp\37a1e54f-60bb-4d03-8364-f09756a14729\VssAdministrator_WIN-TOJS93SFU8U_2020-11-03.22_21_21.coveragexml

What I already Tried

1. Upgrade from .NET Core 3.0 to 3.1
2. Update all NuGet Packages
3. Setting sonar.cs.vscovaragexml.reportsPath
4. Removing sonar.cs.vstest.reportsPaths

Pipeline Snippet

pool:
  vmImage: 'windows-latest'

variables:
  buildConfiguration: 'Release'

steps:
- task: DotNetCoreCLI@2
  displayName: 'Restore project dependencies'
  inputs:
    command: 'restore'
    projects: '**/*.csproj'

- task: SonarCloudPrepare@1
  inputs:
    SonarCloud: '<value removed>'
    organization: '<value removed>'
    scannerMode: 'MSBuild'
    projectKey: '<value removed>'
    projectName: '<value removed>'
    extraProperties: 'sonar.cs.vstest.reportsPaths=$(Agent.TempDirectory)/*.trx'

- task: DotNetCoreCLI@2
  displayName: 'Build the project - $(buildConfiguration)'
  inputs:
    command: 'build'
    arguments: '--no-restore --configuration $(buildConfiguration)'
    projects: '**/*.csproj'

- task: DotNetCoreCLI@2
  displayName: 'Run unittests'
  inputs:
    command: test
    projects: '**/*.csproj'
    arguments: '--configuration $(buildConfiguration) --collect "Code coverage"'

- task: SonarCloudAnalyze@1

- task: SonarCloudPublish@1
  inputs:
    pollingTimeoutSec: '300'

[zamojski]

@jbocc we're facing the same issue on SonarCloud.

[mickael-caro-sonarsource]

Hi there, i finally rolled back the Azure devops extension with the Scanner for MSBuild version 4.10. This bug i'm currently dealing which requests a deep dive investigation, which will follow tomorrow morning, UTC time.

Thanks and sorry again for the incovenience.

[MaxChal]

Hi,

I can confirm that the temporary rollback has fix our pipelines.

[xtroncode]

Hi,

Will this be fixed in future releases ?

Thanks.

[ivanch]

Had the same problem, used "Debug" instead of "Release" compile mode and the problem was fixed.