Code Coverage report is no longer imported

[hangar18rip]

Description

Using this version of the C# plugin in combination with latest SonarQube version (6.4), I cannot import CodeCoverage from VSTS Using the same build definition with an older version of the plugin 5.10.1 (build 1411) and an older version of SonarQube (6.3.1), I have coverage

Repro steps

• Create a standard VSTS build for .Net and add SonarQube tasks (2.0 or 3.0) on a .Net project with coverage enabled
• Install C# plugin in version 5.10.1 (build 1411)
• Run the analysis, Code Coverage should appear in SonarQube project page
• Upgrade C# plugin to version 6.0 (build 2033)
• Run the analysis same analysis again
• Code Coverage should not appear anymore in SonarQube

Expected behavior

CoverageXml file exists and should be used for the analysis and appear in SonarQube project page

Actual behavior

CoverageXml file is not used and no coverage appears in SonarQube

Known workarounds

Use a previous version of this plugin. Tested with 5.10.1 (build 1411)

Related information

• SonarC# 6.0 (build 2033)
• Visual Studio 2015
• VSTS SonarQube tasks, version 2 and 3

[Evangelink]

Hi @hangar18rip,

Quick question, you are talking about SonarC# 5.10.1 against SQ 6.3.1 and SonarC# against SQ 6.4. Could you make an inverted test (i.e. SonarC# 5.10.1 to SQ 6.4 and SonarC# to SQ 6.3?).

Thanks

[hangar18rip]

Hi @Evangelink, I have downgraded the C# plugin to 5.11 on a 6.4 version of SonarQube. This is the setup where we have detected the issue I can try to upgrade to 6.0 on my older version of SonarQube to give it a try

[hangar18rip]

The 6.0 version seems to be OK with SonarQube 6.3.1

[Evangelink]

Do you use the *.it.reportPaths= or the *.reportPaths= syntax for your coverage report? Also, could you provide the log of the analysis with SQ 6.4 and SonarC# 6.0?

[hangar18rip]

I don't provide any additional parameter, so no, I am not using .it.reportPaths= or .reportPaths= I send you the logs as soon as I get back home

[hangar18rip]

I will reinstall the v6.0 version tomorrow. I already have the log of the v5.11. I will upload both then

[hangar18rip]

Coverage OK - Full log.zip Coverage KO - Full log.zip

[hangar18rip]

@Evangelink I have updated to 6.1 to test if the issue is fixed. It is not

[warlof]

I have the same issue since using plugin 6± package on sonar 6.4. No more analysis are pushed.

I've rollback to 5.11 and it's working well.

FYI, i've following warns with 6+ plugins :

14:09:31.980 WARN  - Protobuf file not found: null\output-cs\encoding.pb
14:09:31.999 WARN  - Protobuf file not found: null\output-cs\metrics.pb
14:09:32.002 WARN  - Protobuf file not found: null\output-cs\issues.pb
14:09:32.005 WARN  - Protobuf file not found: null\output-cs\token-type.pb
14:09:32.008 WARN  - Protobuf file not found: null\output-cs\symrefs.pb
14:09:32.010 WARN  - Protobuf file not found: null\output-cs\token-cpd.pb

Using both cli scanner and msbuild scanner

[ydbendasan]

I'm installing a fresh instance of SonarQube on my environment and I have the same issue. The Docker Image version which uses an older version of C# plugin works fine.

6.1(build 2359) doesn't work 5.10.1 (build 1411) works

both on SQ 6.4(build 25310)

[Evangelink]

Hi guys @hangar18rip @warlof @ydbendasan ,

I have made a quick test locally with a SQ server on 6.4, SonarC# 6.2 and Scanner for MSBuild 3.0.1.635 and my coverage is correctly imported. I have a simple console application with a simple test project and I am running the following commands:

scanner begin /k:coverage-import /d:sonar.cs.vscoveragexml.reportsPaths=**/*.coveragexml
msbuild /t:rebuild
"%VSINSTALLDIR%\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe" collect /output:"%CD%\VisualStudio.coverage" "%VSINSTALLDIR%\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" "UnitTestProject1\bin\Debug\UnitTestProject1.dll"
"%VSINSTALLDIR%\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe" analyze /output:"%CD%\VisualStudio.coveragexml" "%CD%\VisualStudio.coverage"
scanner end

Could you tell me what you are doing differently? Could you manage to create a small reproducer of the issue so I can debug it?

[hangar18rip]

Hi @Evangelink, I have deployed the 6.2 update to the SonarQube server where I have detected the issue on 6.0 and the issue is still present. We run the analysis during a VSTS build. The MSBuild Scanner version is 3.0.1.778 in the log (C:\agent_work_tasks\SonarQubeScannerMsBuildBegin_15b84ca1-b62f-4a2a-a403-89b77a063157\3.0.1\SonarQubeScannerMsBuild\sonar-scanner-3.0.3.778\bin..\conf\sonar-scanner.properties) I will be off the next weeks so I can't help you much on testing on other projects

[Evangelink]

Have you checked the Code Coverage Enabled on the Visual Studio Test task? Also are you providing a path to the coveragexml file or are you relying on the script to detect the path?

[hangar18rip]

I don't. The only thing I change is the plugin. 5.1 works, 6.* works but doesn't collect the coverage

[Evangelink]

Can you make a quick test by just checking this Code Coverage Enabled in Visual Studio Test task?

[hangar18rip]

The Code Coverage is already activated and is always uploaded in build result. No issue on this side

[Evangelink]

Are you working onto some Open Source project that I could fork/clone to make test? If not and if the project is not too big would you be allowed to mail it to me?

[hangar18rip]

I can't, sorry

[jvr84]

I also have the same issue:

i found in my build log the following difference:

6.2 DEBUG: Sensors : C# -> SonarJavaXmlFileSensor

5.11 DEBUG: Sensors : C# -> C# Unit Tests Coverage Report Import -> C# Integration Tests Coverage Report Import -> C# Unit Test Results Import -> SonarJavaXmlFileSensor INFO: Sensor C# Unit Tests Coverage Report Import [csharp] INFO: Parsing the Visual Studio coverage XML report ...coveragexml INFO: Adding this code coverage report to the cache for later reuse: ...coveragexml

Im using: SonarQube Scanner 3.0.3.778 SonarQube server 6.4.0 C# 5.11.0.1761 (csharp)

[Evangelink]

@jvr84 Just to be sure, using SonarC# 5.11 on SQ 6.4 is working but using SonarC# 6.2 on SQ 6.4 isn't?

[jvr84]

correct

[Evangelink]

If one of you could provide me some small reproducer that would be great because whatever I do I cannot reproduce this issue.

[hangar18rip]

@Evangelink You try to reproduce it from the command line ? Can you give it a try using VSTS ? Maybe a contextual issue due to some kind of special context (env var, tooling) during the build

[jvr84]

We are using Version 6.4 (build 25310) We had this issue with vsts and older xaml builds. both running on different machines

[warlof]

I'm not using sonar with tfs yet, so I'm not sure it's related. I'll make further tests and attempt to provide you a test project.

I saw you were using an older scanner version. Could it be related ?

[Evangelink]

@warlof Which version are you using (it's not in your previous post)? Regarding the scanner cli, it is not the recommended way to analyzer C#/VB.Net project we do recommend to use the scanner for MSBuild. Also, I am not sure your issue is the same as the others. In your case no info are pushed (empty protobuf files), in theirs it is only a problem of Code Coverage. Have you seen that we dropped the support for MSBuild 12 from SonarC# 6.0 and I think your issue is related to that.

[performation]

I have the same issue, using TFS 2017, which shows Code Coverage results, but it's missing in SonarQube.

Version info:

• TFS 2017 Update 2 (server A)
• SonarQube 6.4 (build 25310) (server B)
• C# plugin 6.2 (build 2536) (server B)

My scanner context file contains the code coverage xml (which exists, I've checked): sonar.cs.vscoveragexml.reportsPaths=D:\NET-D\1_work\6\s\TestResults\sa.tfs-nlsvptfs01_NLSVPTFS02 2017-08-03 10_10_43\In\NLSVPTFS02\sa.tfs-nlsvptfs01_NLSVPTFS02 2017-08-03 10_10_37.coveragexml

[performation]

I've downgraded to C# plugin 5.11 (build 1761) from my old install, and I can confirm that Code Coverage results are back

[warlof]

@Evangelink I'm using SonarQube 6.4 and rolledback to Sonar-CSharp 5.11 used with both cli-scanner 3.0.3 and msbuild-scanner 3.0.0.629 without issues. But since the sonar-csharp 6.0, nothing appear on the metrics.

I've saw you add a 6.3 build which I've didn't tried yet. Same for Sonar 6.5.

What can I provide you in order to help to better digg into that issue except for project source code ? (I've again forget my laptop at the office, so can't provide much more information yet :()

EDIT : It seems the couple Sonar 6.5 + MSBuild-Scanner 3.0.1 + sonar-csharp 5.10.1 is working well. I'll test tomorrow with sonar-csharp 6.2

[Evangelink]

Hi all, I have reproduced the issue using the latest VSTS extension. It seems like we have a regression on the Scanner for MSBuild (ticket: https://jira.sonarsource.com/browse/SONARMSBRU-339).

While this ticket is being addressed you can fix this issue by simply adding /d:sonar.cs.vscoveragexml.reportsPaths="**\*.coveragexml" into the Additional Settings of the SonarQube Scanner for MSBuild - Begin Analysis (new) task.

I am closing this thread as I have provided a workaround and created a ticket.

[uskuandeng]

@Evangelink I got an error of "Warning: No test is available in C:\Bamboo_bld\xml-data\build-dir\SEPLC-SONARQUBEICS-SQT\ICS\Instruments\SM\AcquitySM\Tests\UnitTests\ConsoleSMTests\bin\Release\ConsoleSMTests.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.". What could be the problem? I have all latest SonarQube and related plugins ; and my VS2015 is upgraded from Pro to Enterprise edition.

[Evangelink]

Hi @uskuandeng, Could you open a new issue for this as it doesn't seem to be directly related to this issue? It seems that the error is purely related to test execution (mstest or vstest) and not at all linked to the scanner for msbuild. Just to be sure, are you using the Scanner for MSBuild directly or through the VSTS extension? Is the error appearing during the scanner end step or during the test execution (I would bet on this one).

[uskuandeng]

@Evangelink I will create one. To answer your questions. I'm using Scanner for MSBuild but through the Wrapper because my project is a mixture of C# and C/++. The error is from the command line (triggered by Bamboo) "%VSINSTALLDIR%Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe" collect /output:"%PROJDIR%\VisualStudio.coverage" "%VTESTCONSOLE%\vstest.console.exe" "%TESTPROJDIR%\ConsoleSMTests.dll". The analyzed project is a legend (old) C#/C/C++ mixture. I have successfully use the same command for a new test project!

[hangar18rip]

Hi there,

@Evangelink : any plan to fix issue in the VSTS Extension ?

Regards,

[tijanacosovic]

I have the same issue with Open cover

[McMlok]

Hi all,

same issue in our enviroment

C# Plugin 6.7 SonarQube server 6.7 TFS 2017 Update 3 on-premise TFS Sonar task 3.1

Thaks @Evangelink for workarround. But before update of C# plugin it works out-of-box