FCC does not show code coverage for sourcelink enabled projects.

dozer75 commented 1 year ago

Installed product versions

Visual Studio: 2022 Community 17.7.0
This extension: 1.1.194

Description

The extension does not create code coverage for dotnet/sourcelink enabled projects out of the package. Coverlet supports this by adding the runtime parameter --use-source-link, however I have no idea how to make it work with MS Code Coverage.

Steps to recreate

Add sourcelink to the class library as described here.
Run the tests so that FCC generates code coverage.

Current behavior

No code coverage is generated for the class library with sourcelink.

Expected behavior

Code coverage result should be generated both with and without sourcelink.

Side Notes

Log for sourcelink enabled class library

Fine Code Coverage 2023-08-14 14:52:10: See option RunMsCodeCoverage for a better ( Beta ) experience.  https://github.com/FortuneN/FineCodeCoverage/blob/master/README.md
Fine Code Coverage 2023-08-14 14:52:10: Coverage collected when tests finish. RunInParallel option true for immediate
Fine Code Coverage 2023-08-14 14:52:12: ================================== START ==================================
Fine Code Coverage 2023-08-14 14:52:12: Copy : <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\KISS.Moq.Logger.dll -> <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\build-output\KISS.Moq.Logger.dll
Copy : <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\KISS.Moq.Logger.Test.deps.json -> <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\build-output\KISS.Moq.Logger.Test.deps.json
Copy : <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\CoverletSourceRootsMapping -> <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\build-output\CoverletSourceRootsMapping
Copy : <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\KISS.Moq.Logger.Test.pdb -> <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\build-output\KISS.Moq.Logger.Test.pdb
Copy : <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\KISS.Moq.Logger.pdb -> <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\build-output\KISS.Moq.Logger.pdb
Copy : <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\KISS.Moq.Logger.Test.dll -> <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\build-output\KISS.Moq.Logger.Test.dll
File synchronization duration : 00:00:00.0189380
Fine Code Coverage 2023-08-14 14:52:12: Copy : <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\KISS.Moq.Logger.pdb -> <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\fine-code-coverage\build-output\KISS.Moq.Logger.pdb
Copy : <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\KISS.Moq.Logger.Test.dll -> <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\fine-code-coverage\build-output\KISS.Moq.Logger.Test.dll
Copy : <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\KISS.Moq.Logger.Test.deps.json -> <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\fine-code-coverage\build-output\KISS.Moq.Logger.Test.deps.json
Copy : <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\KISS.Moq.Logger.Test.pdb -> <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\fine-code-coverage\build-output\KISS.Moq.Logger.Test.pdb
Copy : <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\KISS.Moq.Logger.dll -> <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\fine-code-coverage\build-output\KISS.Moq.Logger.dll
Copy : <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\CoverletSourceRootsMapping -> <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\fine-code-coverage\build-output\CoverletSourceRootsMapping
File synchronization duration : 00:00:00.0185095
Fine Code Coverage 2023-08-14 14:52:12: Run Coverlet (Moq.Logger.Test (net7.0))
Fine Code Coverage 2023-08-14 14:52:12: Coverlet Run (Moq.Logger.Test (net7.0)) Arguments 
"<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\build-output\KISS.Moq.Logger.Test.dll"
--format "cobertura"
--exclude-by-file "**/Migrations/*"
--exclude-by-attribute "GeneratedCode"
--target "dotnet"
--threshold-type line
--threshold-stat total
--threshold 0
--output "<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Moq.Logger.Test (net7.0).coverage.xml"
--targetargs "test  ""<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\build-output\KISS.Moq.Logger.Test.dll"" --nologo --blame  --results-directory ""<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output"" --diag ""<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output/diagnostics.log""  "
Fine Code Coverage 2023-08-14 14:52:14: Coverlet Run (Moq.Logger.Test (net7.0))
Starting test execution, please wait...
Logging Vstest Diagnostics in file: <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\diagnostics.log
A total of 1 test files matched the specified pattern.
Passed!  - Failed:     0, Passed:     2, Skipped:     0, Total:     2, Duration: 14 ms - KISS.Moq.Logger.Test.dll (net7.0)

Calculating coverage result...
  Generating report '<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Moq.Logger.Test (net7.0).coverage.xml'
+--------+------+--------+--------+
| Module | Line | Branch | Method |
+--------+------+--------+--------+

+---------+------+--------+--------+
|         | Line | Branch | Method |
+---------+------+--------+--------+
| Total   | 0%   | 0%     | 0%     |
+---------+------+--------+--------+
| Average | 0%   | 0%     | 0%     |
+---------+------+--------+--------+
Fine Code Coverage 2023-08-14 14:52:14: Completed coverage for (Moq.Logger.Test (net7.0)) : 00:00:02.1615721
Fine Code Coverage 2023-08-14 14:52:14: Run Coverlet (Moq.Logger.Test (net6.0))
Fine Code Coverage 2023-08-14 14:52:15: Coverlet Run (Moq.Logger.Test (net6.0)) Arguments 
"<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\fine-code-coverage\build-output\KISS.Moq.Logger.Test.dll"
--format "cobertura"
--exclude-by-file "**/Migrations/*"
--exclude-by-attribute "GeneratedCode"
--target "dotnet"
--threshold-type line
--threshold-stat total
--threshold 0
--output "<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\fine-code-coverage\coverage-tool-output\Moq.Logger.Test (net6.0).coverage.xml"
--targetargs "test  ""<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\fine-code-coverage\build-output\KISS.Moq.Logger.Test.dll"" --nologo --blame  --results-directory ""<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\fine-code-coverage\coverage-tool-output"" --diag ""<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\fine-code-coverage\coverage-tool-output/diagnostics.log""  "
Fine Code Coverage 2023-08-14 14:52:17: Coverlet Run (Moq.Logger.Test (net6.0))
Starting test execution, please wait...
Logging Vstest Diagnostics in file: <project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\fine-code-coverage\coverage-tool-output\diagnostics.log
A total of 1 test files matched the specified pattern.
Passed!  - Failed:     0, Passed:     2, Skipped:     0, Total:     2, Duration: 18 ms - KISS.Moq.Logger.Test.dll (net6.0)

Calculating coverage result...
  Generating report '<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\fine-code-coverage\coverage-tool-output\Moq.Logger.Test (net6.0).coverage.xml'
+--------+------+--------+--------+
| Module | Line | Branch | Method |
+--------+------+--------+--------+

+---------+------+--------+--------+
|         | Line | Branch | Method |
+---------+------+--------+--------+
| Total   | 0%   | 0%     | 0%     |
+---------+------+--------+--------+
| Average | 0%   | 0%     | 0%     |
+---------+------+--------+--------+
Fine Code Coverage 2023-08-14 14:52:17: Completed coverage for (Moq.Logger.Test (net6.0)) : 00:00:02.4656310
Fine Code Coverage 2023-08-14 14:52:17: ReportGenerator Run Arguments [reporttype:Cobertura] 
"-targetdir:<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output"
"-reports:<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Moq.Logger.Test (net7.0).coverage.xml;<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\fine-code-coverage\coverage-tool-output\Moq.Logger.Test (net6.0).coverage.xml"
"-reporttypes:Cobertura"
Fine Code Coverage 2023-08-14 14:52:17: ReportGenerator Run [reporttype:Cobertura]
2023-08-14T14:52:17: Arguments
2023-08-14T14:52:17:  -targetdir:<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output
2023-08-14T14:52:17:  -reports:<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Moq.Logger.Test (net7.0).coverage.xml;<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net6.0\fine-code-coverage\coverage-tool-output\Moq.Logger.Test (net6.0).coverage.xml
2023-08-14T14:52:17:  -reporttypes:Cobertura
2023-08-14T14:52:17: Writing report file '<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Cobertura.xml'
2023-08-14T14:52:17: Report generation took 0,1 seconds
Fine Code Coverage 2023-08-14 14:52:18: ReportGenerator Run Arguments [reporttype:HtmlInline_AzurePipelines] 
"-targetdir:<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output"
"-reports:<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Cobertura.xml"
"-plugins:<plugin folder>\ReportGeneratorPlugins.dll"
"-reporttypes:FccLight"
"riskHotspotsAnalysisThresholds:metricThresholdForCyclomaticComplexity=30"
"riskHotspotsAnalysisThresholds:metricThresholdForCrapScore=15"
"riskHotspotsAnalysisThresholds:metricThresholdForNPathComplexity=200"
Fine Code Coverage 2023-08-14 14:52:18: ReportGenerator Run [reporttype:HtmlInline_AzurePipelines]
2023-08-14T14:52:18: Arguments
2023-08-14T14:52:18:  -targetdir:<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output
2023-08-14T14:52:18:  -reports:<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Cobertura.xml
2023-08-14T14:52:18:  -plugins:<extension folder>\3ducdhei.pv1\ReportGeneratorPlugins.dll
2023-08-14T14:52:18:  -reporttypes:FccLight
2023-08-14T14:52:18:  riskHotspotsAnalysisThresholds:metricThresholdForCyclomaticComplexity=30
2023-08-14T14:52:18:  riskHotspotsAnalysisThresholds:metricThresholdForCrapScore=15
2023-08-14T14:52:18:  riskHotspotsAnalysisThresholds:metricThresholdForNPathComplexity=200
2023-08-14T14:52:18: Writing report file '<project folder>\Moq.Logger\test\Moq.Logger.Test\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\index.html'
2023-08-14T14:52:18: Report generation took 0,1 seconds
Fine Code Coverage 2023-08-14 14:52:18: Processing cobertura
Fine Code Coverage 2023-08-14 14:52:18: Processing report
Fine Code Coverage 2023-08-14 14:52:18: ================================== DONE ==================================

tonyhallett commented 1 year ago

Do you get results if you remove sourcelink ?

dozer75 commented 1 year ago

Do you get results if you remove sourcelink ?

Yes

tonyhallett commented 1 year ago

I am not sure if the absence of --use-source-link is the issue.

When you specify the --use-source-link flag, Coverlet will generate results that contain the URL to the source files in your source control instead of local file paths.

This suggests to me that it should work without the flag. If the paths changed to URLs then FCC will not be able to provide the file highlighting.

Can you check the diagnostic log. Perhaps there is further information there.

dozer75 commented 1 year ago

I found what the issue is.

I had (for some strange reason) the property ContinuousIntegrationBuild set in my projects csproj file... This property seems to cause issues with the combination of the Sourcelink package, Coverlet and FCC. Removing the property from the csproj file (which shouldn't be there anyway) solved the issue.

The property ContinuousIntegrationBuild is only needed while running CI build engines anyway so it is more correct to set it in build scripts. The bulid scripts themselves can set the UseSourceLink (or --use-source-link) for generating code coverage using CI builds.

tonyhallett commented 1 year ago

I had come across ContinuousIntegrationBuild in the coverlet issues. Was going to suggest that at some point

dozer75 commented 1 year ago

I had come across ContinuousIntegrationBuild in the coverlet issues. Was going to suggest that at some point

Yes, it was comments around that which caught my attention of it.. However, that works together with the UseSourceLink option and generates code coverage, but after investigating the resulting report I don't think it would be of help anyway since that adds the Git commit URL to the source file and thus it would not work well together with source lookup in FCC (possible of course, but would be a ugly hack)...

At least now it is recorded in an issue here as well that ContinuousIntegrationBuild interferes with FCC logic, and it is not necessary to have in the project file anyway so there is a workaround.

FortuneN / FineCodeCoverage