search

FortuneN / FineCodeCoverage

Visualize unit test code coverage easily for free in Visual Studio Community Edition (and other editions too)
https://marketplace.visualstudio.com/items?itemName=FortuneNgwenya.FineCodeCoverage
Other
516 stars 39 forks source link

No Results when using MS Code Coverage and Running 2 Test assemblies #334

Closed bb-avi closed 1 year ago

bb-avi commented 1 year ago

Installed product versions

Description

I've setup MS Code Coverage and it works properly when I run each test assembly on its own. But when I try to run both test assemblies in the same run, then no output is generated and there are Object reference not set to an instance of an object error messages in the log

Steps to recreate

  1. Setup to use MS Code Coverage (with *.runsettings file)
  2. In Test Explorer run Unit Tests - Fine Code Coverage shows results
  3. In Test Explorer run Integration Tests - Fine Code Coverage shows results
  4. In Test Explorer run both Integration Tests and Unit Tests - Fine Code Coverage does NOT show results and there are errors in the FCC log
  5. Setup to use Coverlet
  6. In Test Explorer run both Integration Tests and Unit Tests - Fine Code Coverage shows results (after a long delay)

Current behavior

It is NOT showing coverage results when both test assemblies (Unit and Integration Tests) are run.

Expected behavior

It should show coverage results when both test assemblies (Unit and Integration Tests) are run.

Notes

I cannot provide the solution since this is proprietary code.

Fine Code Coverage Settings Modified

Exclude / Include (Microsoft) > ModulePathsExclude

Run (Common) > RunMsCodeCoverage

Logs

Failed Combined Run Log using Ms Code Coverage ``` Fine Code Coverage 6/23/2023 12:27:50 PM: Ms code coverage Fine Code Coverage 6/23/2023 12:28:47 PM: ReportGenerator Run Arguments [reporttype:Cobertura] "-targetdir:C:\company\git\dotnet\Packages\[REDACTED]\tests\[REDACTED].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output" "-reports:C:\company\git\dotnet\Packages\[REDACTED]\tests\[REDACTED].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\c50ed9ad-c418-4dac-92b4-e2391ba106ce\devUser_dotnetvm_2023-06-23.12_27_51.cobertura.xml" "-reporttypes:Cobertura" Fine Code Coverage 6/23/2023 12:28:48 PM: ReportGenerator Run [reporttype:Cobertura] 2023-06-23T12:28:48: Arguments 2023-06-23T12:28:48: -targetdir:C:\company\git\dotnet\Packages\[REDACTED]\tests\[REDACTED].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output 2023-06-23T12:28:48: -reports:C:\company\git\dotnet\Packages\[REDACTED]\tests\[REDACTED].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\c50ed9ad-c418-4dac-92b4-e2391ba106ce\devUser_dotnetvm_2023-06-23.12_27_51.cobertura.xml 2023-06-23T12:28:48: -reporttypes:Cobertura 2023-06-23T12:28:48: Error during reading report 'C:\company\git\dotnet\Packages\[REDACTED]\tests\[REDACTED].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\c50ed9ad-c418-4dac-92b4-e2391ba106ce\devUser_dotnetvm_2023-06-23.12_27_51.cobertura.xml' (Size: 195.4KB): Multiple errors Object reference not set to an instance of an object. Object reference not set to an instance of an object. Object reference not set to an instance of an object. 2023-06-23T12:28:48: Writing report file 'C:\company\git\dotnet\Packages\[REDACTED]\tests\[REDACTED].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Cobertura.xml' 2023-06-23T12:28:48: Report generation took 0.2 seconds Fine Code Coverage 6/23/2023 12:28:48 PM: ReportGenerator Run Arguments [reporttype:HtmlInline_AzurePipelines] "-targetdir:C:\company\git\dotnet\Packages\[REDACTED]\tests\[REDACTED].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output" "-reports:C:\company\git\dotnet\Packages\[REDACTED]\tests\[REDACTED].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Cobertura.xml" "-plugins:c:\users\devUser\appdata\local\microsoft\visualstudio\17.0_38fa87c3\extensions\vz4drnyg.0wf\ReportGeneratorPlugins.dll" "-reporttypes:FccLight" "riskHotspotsAnalysisThresholds:metricThresholdForCyclomaticComplexity=30" "riskHotspotsAnalysisThresholds:metricThresholdForCrapScore=15" "riskHotspotsAnalysisThresholds:metricThresholdForNPathComplexity=200" Fine Code Coverage 6/23/2023 12:28:49 PM: ReportGenerator Run [reporttype:HtmlInline_AzurePipelines] 2023-06-23T12:28:48: Arguments 2023-06-23T12:28:48: -targetdir:C:\company\git\dotnet\Packages\[REDACTED]\tests\[REDACTED].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output 2023-06-23T12:28:48: -reports:C:\company\git\dotnet\Packages\[REDACTED]\tests\[REDACTED].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Cobertura.xml 2023-06-23T12:28:48: -plugins:c:\users\devUser\appdata\local\microsoft\visualstudio\17.0_38fa87c3\extensions\vz4drnyg.0wf\ReportGeneratorPlugins.dll 2023-06-23T12:28:48: -reporttypes:FccLight 2023-06-23T12:28:48: riskHotspotsAnalysisThresholds:metricThresholdForCyclomaticComplexity=30 2023-06-23T12:28:48: riskHotspotsAnalysisThresholds:metricThresholdForCrapScore=15 2023-06-23T12:28:48: riskHotspotsAnalysisThresholds:metricThresholdForNPathComplexity=200 2023-06-23T12:28:49: Writing report file 'C:\company\git\dotnet\Packages\[REDACTED]\tests\[REDACTED].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\index.html' 2023-06-23T12:28:49: Report generation took 0.2 seconds Fine Code Coverage 6/23/2023 12:28:49 PM: Processing cobertura Fine Code Coverage 6/23/2023 12:28:49 PM: Processing report Fine Code Coverage 6/23/2023 12:28:49 PM: ================================== DONE ================================== ```
Successful run of Unit Tests Log using Ms Code Coverage ``` Fine Code Coverage 6/23/2023 12:35:31 PM: Ms code coverage Fine Code Coverage 6/23/2023 12:35:39 PM: ReportGenerator Run Arguments [reporttype:Cobertura] "-targetdir:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output" "-reports:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\1ac4b6ee-4ee9-4831-907e-3b1db6a84b5b\devUser_dotnetvm_2023-06-23.12_35_33.cobertura.xml" "-reporttypes:Cobertura" Fine Code Coverage 6/23/2023 12:35:40 PM: ReportGenerator Run [reporttype:Cobertura] 2023-06-23T12:35:40: Arguments 2023-06-23T12:35:40: -targetdir:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output 2023-06-23T12:35:40: -reports:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\1ac4b6ee-4ee9-4831-907e-3b1db6a84b5b\devUser_dotnetvm_2023-06-23.12_35_33.cobertura.xml 2023-06-23T12:35:40: -reporttypes:Cobertura 2023-06-23T12:35:40: Writing report file 'C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Cobertura.xml' 2023-06-23T12:35:40: Report generation took 0.4 seconds Fine Code Coverage 6/23/2023 12:35:40 PM: ReportGenerator Run Arguments [reporttype:HtmlInline_AzurePipelines] "-targetdir:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output" "-reports:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Cobertura.xml" "-plugins:c:\users\devUser\appdata\local\microsoft\visualstudio\17.0_38fa87c3\extensions\vz4drnyg.0wf\ReportGeneratorPlugins.dll" "-reporttypes:FccLight" "riskHotspotsAnalysisThresholds:metricThresholdForCyclomaticComplexity=30" "riskHotspotsAnalysisThresholds:metricThresholdForCrapScore=15" "riskHotspotsAnalysisThresholds:metricThresholdForNPathComplexity=200" Fine Code Coverage 6/23/2023 12:35:41 PM: ReportGenerator Run [reporttype:HtmlInline_AzurePipelines] 2023-06-23T12:35:41: Arguments 2023-06-23T12:35:41: -targetdir:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output 2023-06-23T12:35:41: -reports:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Cobertura.xml 2023-06-23T12:35:41: -plugins:c:\users\devUser\appdata\local\microsoft\visualstudio\17.0_38fa87c3\extensions\vz4drnyg.0wf\ReportGeneratorPlugins.dll 2023-06-23T12:35:41: -reporttypes:FccLight 2023-06-23T12:35:41: riskHotspotsAnalysisThresholds:metricThresholdForCyclomaticComplexity=30 2023-06-23T12:35:41: riskHotspotsAnalysisThresholds:metricThresholdForCrapScore=15 2023-06-23T12:35:41: riskHotspotsAnalysisThresholds:metricThresholdForNPathComplexity=200 2023-06-23T12:35:41: Writing report file 'C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\index.html' 2023-06-23T12:35:41: Report generation took 0.3 seconds Fine Code Coverage 6/23/2023 12:35:41 PM: Processing cobertura Fine Code Coverage 6/23/2023 12:35:41 PM: Processing report Fine Code Coverage 6/23/2023 12:35:41 PM: ================================== DONE ================================== ```
Successful run of Integration Tests Log using Ms Code Coverage ``` Fine Code Coverage 6/23/2023 12:47:23 PM: Ms code coverage Fine Code Coverage 6/23/2023 12:48:08 PM: ReportGenerator Run Arguments [reporttype:Cobertura] "-targetdir:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output" "-reports:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\ffa4d13e-b258-4312-a938-b1ae37c6f191\devUser_dotnetvm_2023-06-23.12_47_25.cobertura.xml" "-reporttypes:Cobertura" Fine Code Coverage 6/23/2023 12:48:09 PM: ReportGenerator Run [reporttype:Cobertura] 2023-06-23T12:48:08: Arguments 2023-06-23T12:48:08: -targetdir:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output 2023-06-23T12:48:08: -reports:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\ffa4d13e-b258-4312-a938-b1ae37c6f191\devUser_dotnetvm_2023-06-23.12_47_25.cobertura.xml 2023-06-23T12:48:08: -reporttypes:Cobertura 2023-06-23T12:48:09: Writing report file 'C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Cobertura.xml' 2023-06-23T12:48:09: Report generation took 0.4 seconds Fine Code Coverage 6/23/2023 12:48:09 PM: ReportGenerator Run Arguments [reporttype:HtmlInline_AzurePipelines] "-targetdir:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output" "-reports:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Cobertura.xml" "-plugins:c:\users\devUser\appdata\local\microsoft\visualstudio\17.0_38fa87c3\extensions\vz4drnyg.0wf\ReportGeneratorPlugins.dll" "-reporttypes:FccLight" "riskHotspotsAnalysisThresholds:metricThresholdForCyclomaticComplexity=30" "riskHotspotsAnalysisThresholds:metricThresholdForCrapScore=15" "riskHotspotsAnalysisThresholds:metricThresholdForNPathComplexity=200" Fine Code Coverage 6/23/2023 12:48:09 PM: ReportGenerator Run [reporttype:HtmlInline_AzurePipelines] 2023-06-23T12:48:09: Arguments 2023-06-23T12:48:09: -targetdir:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output 2023-06-23T12:48:09: -reports:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Cobertura.xml 2023-06-23T12:48:09: -plugins:c:\users\devUser\appdata\local\microsoft\visualstudio\17.0_38fa87c3\extensions\vz4drnyg.0wf\ReportGeneratorPlugins.dll 2023-06-23T12:48:09: -reporttypes:FccLight 2023-06-23T12:48:09: riskHotspotsAnalysisThresholds:metricThresholdForCyclomaticComplexity=30 2023-06-23T12:48:09: riskHotspotsAnalysisThresholds:metricThresholdForCrapScore=15 2023-06-23T12:48:09: riskHotspotsAnalysisThresholds:metricThresholdForNPathComplexity=200 2023-06-23T12:48:09: Writing report file 'C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\index.html' 2023-06-23T12:48:09: Report generation took 0.4 seconds Fine Code Coverage 6/23/2023 12:48:09 PM: Processing cobertura Fine Code Coverage 6/23/2023 12:48:09 PM: Processing report Fine Code Coverage 6/23/2023 12:48:10 PM: ================================== DONE ================================== ```
Successful Combined Run Log using Coverlet I just flipped the RunMsCodeCoverage setting to `No`. ``` Fine Code Coverage 6/23/2023 1:33:50 PM: See option RunMsCodeCoverage for a better ( Beta ) experience. https://github.com/FortuneN/FineCodeCoverage/blob/master/README.md Fine Code Coverage 6/23/2023 1:33:50 PM: Coverage collected when tests finish. RunInParallel option true for immediate Fine Code Coverage 6/23/2023 1:34:29 PM: ================================== START ================================== Fine Code Coverage 6/23/2023 1:34:30 PM: Copy : C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\nunit_random_seed.tmp -> C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\build-output\nunit_random_seed.tmp Copy : C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\[Redacted].UnitTests.dll -> C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\build-output\[Redacted].UnitTests.dll Copy : C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\[Redacted].UnitTests.pdb -> C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\build-output\[Redacted].UnitTests.pdb Copy : C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\[Redacted].UnitTests.deps.json -> C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\build-output\[Redacted].UnitTests.deps.json File synchronization duration : 00:00:00.0511821 Fine Code Coverage 6/23/2023 1:34:30 PM: Copy : C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\[Redacted].IntegrationTests.deps.json -> C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\build-output\[Redacted].IntegrationTests.deps.json Copy : C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\[Redacted].IntegrationTests.dll -> C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\build-output\[Redacted].IntegrationTests.dll Copy : C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\nunit_random_seed.tmp -> C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\build-output\nunit_random_seed.tmp Copy : C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\[Redacted].IntegrationTests.pdb -> C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\build-output\[Redacted].IntegrationTests.pdb File synchronization duration : 00:00:00.0497232 Fine Code Coverage 6/23/2023 1:34:30 PM: Run Coverlet ([Redacted].UnitTests) Fine Code Coverage 6/23/2023 1:34:30 PM: Coverlet Run ([Redacted].UnitTests) Arguments "C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\build-output\[Redacted].UnitTests.dll" --format "cobertura" --exclude-by-file "**/Migrations/*" --exclude-by-attribute "GeneratedCode" --target "dotnet" --threshold-type line --threshold-stat total --threshold 0 --output "C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\[Redacted].UnitTests.coverage.xml" --targetargs "test ""C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\build-output\[Redacted].UnitTests.dll"" --nologo --blame --results-directory ""C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output"" --diag ""C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output/diagnostics.log"" " Fine Code Coverage 6/23/2023 1:34:36 PM: Coverlet Run ([Redacted].UnitTests) Starting test execution, please wait... Logging Vstest Diagnostics in file: C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\diagnostics.log A total of 1 test files matched the specified pattern. Passed! - Failed: 0, Passed: 141, Skipped: 0, Total: 141, Duration: 2 s - [Redacted].UnitTests.dll (net7.0) Calculating coverage result... Generating report 'C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\[Redacted].UnitTests.coverage.xml' +------------------+--------+--------+--------+ | Module | Line | Branch | Method | +------------------+--------+--------+--------+ | [Redacted] | 90.25% | 84.12% | 88.05% | +------------------+--------+--------+--------+ +---------+--------+--------+--------+ | | Line | Branch | Method | +---------+--------+--------+--------+ | Total | 90.25% | 84.12% | 88.05% | +---------+--------+--------+--------+ | Average | 90.25% | 84.12% | 88.05% | +---------+--------+--------+--------+ Fine Code Coverage 6/23/2023 1:34:36 PM: Completed coverage for ([Redacted].UnitTests) : 00:00:06.4397385 Fine Code Coverage 6/23/2023 1:34:36 PM: Run Coverlet ([Redacted].IntegrationTests) Fine Code Coverage 6/23/2023 1:34:36 PM: Coverlet Run ([Redacted].IntegrationTests) Arguments "C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\build-output\[Redacted].IntegrationTests.dll" --format "cobertura" --exclude-by-file "**/Migrations/*" --exclude-by-attribute "GeneratedCode" --target "dotnet" --threshold-type line --threshold-stat total --threshold 0 --output "C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\[Redacted].IntegrationTests.coverage.xml" --targetargs "test ""C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\build-output\[Redacted].IntegrationTests.dll"" --nologo --blame --results-directory ""C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output"" --diag ""C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output/diagnostics.log"" " Fine Code Coverage 6/23/2023 1:35:15 PM: Coverlet Run ([Redacted].IntegrationTests) Starting test execution, please wait... Logging Vstest Diagnostics in file: C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\diagnostics.log A total of 1 test files matched the specified pattern. Skipped GetApiTokenStateAsync_UsingSsoToken_ReturnsValidSsoToken [< 1 ms] Passed! - Failed: 0, Passed: 45, Skipped: 1, Total: 46, Duration: 33 s - [Redacted].IntegrationTests.dll (net7.0) Calculating coverage result... Generating report 'C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\[Redacted].IntegrationTests.coverage.xml' +------------------+-------+--------+--------+ | Module | Line | Branch | Method | +------------------+-------+--------+--------+ | [Redacted] | 86.7% | 52.02% | 84.07% | +------------------+-------+--------+--------+ +---------+-------+--------+--------+ | | Line | Branch | Method | +---------+-------+--------+--------+ | Total | 86.7% | 52.02% | 84.07% | +---------+-------+--------+--------+ | Average | 86.7% | 52.02% | 84.07% | +---------+-------+--------+--------+ Fine Code Coverage 6/23/2023 1:35:15 PM: Completed coverage for ([Redacted].IntegrationTests) : 00:00:39.0148791 Fine Code Coverage 6/23/2023 1:35:15 PM: ReportGenerator Run Arguments [reporttype:Cobertura] "-targetdir:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output" "-reports:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\[Redacted].UnitTests.coverage.xml;C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\[Redacted].IntegrationTests.coverage.xml" "-reporttypes:Cobertura" Fine Code Coverage 6/23/2023 1:35:16 PM: ReportGenerator Run [reporttype:Cobertura] 2023-06-23T13:35:15: Arguments 2023-06-23T13:35:15: -targetdir:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output 2023-06-23T13:35:15: -reports:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\[Redacted].UnitTests.coverage.xml;C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].IntegrationTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\[Redacted].IntegrationTests.coverage.xml 2023-06-23T13:35:15: -reporttypes:Cobertura 2023-06-23T13:35:16: Writing report file 'C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Cobertura.xml' 2023-06-23T13:35:16: Report generation took 0.3 seconds Fine Code Coverage 6/23/2023 1:35:16 PM: ReportGenerator Run Arguments [reporttype:HtmlInline_AzurePipelines] "-targetdir:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output" "-reports:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Cobertura.xml" "-plugins:c:\users\devUser\appdata\local\microsoft\visualstudio\17.0_38fa87c3\extensions\vz4drnyg.0wf\ReportGeneratorPlugins.dll" "-reporttypes:FccLight" "riskHotspotsAnalysisThresholds:metricThresholdForCyclomaticComplexity=30" "riskHotspotsAnalysisThresholds:metricThresholdForCrapScore=15" "riskHotspotsAnalysisThresholds:metricThresholdForNPathComplexity=200" Fine Code Coverage 6/23/2023 1:35:17 PM: ReportGenerator Run [reporttype:HtmlInline_AzurePipelines] 2023-06-23T13:35:16: Arguments 2023-06-23T13:35:16: -targetdir:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output 2023-06-23T13:35:16: -reports:C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\Cobertura.xml 2023-06-23T13:35:16: -plugins:c:\users\devUser\appdata\local\microsoft\visualstudio\17.0_38fa87c3\extensions\vz4drnyg.0wf\ReportGeneratorPlugins.dll 2023-06-23T13:35:16: -reporttypes:FccLight 2023-06-23T13:35:16: riskHotspotsAnalysisThresholds:metricThresholdForCyclomaticComplexity=30 2023-06-23T13:35:16: riskHotspotsAnalysisThresholds:metricThresholdForCrapScore=15 2023-06-23T13:35:16: riskHotspotsAnalysisThresholds:metricThresholdForNPathComplexity=200 2023-06-23T13:35:17: Writing report file 'C:\company\git\dotnet\Packages\[Redacted]\tests\[Redacted].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\index.html' 2023-06-23T13:35:17: Report generation took 0.4 seconds Fine Code Coverage 6/23/2023 1:35:17 PM: Processing cobertura Fine Code Coverage 6/23/2023 1:35:17 PM: Processing report Fine Code Coverage 6/23/2023 1:35:18 PM: ================================== DONE ================================== ```

Files

*.runsettings ``` 0 x64 10000 true .\src\*/ModulePath> True True True False ​ ​ ​ quiet foo.trx foo.html True false False False ```
tonyhallett commented 1 year ago

Was the cobertura file present

'C:\company\git\dotnet\Packages[REDACTED]\tests[REDACTED].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\c50ed9ad-c418-4dac-92b4-e2391ba106ce\devUser_dotnetvm_2023-06-23.12_27_51.cobertura.xml'

Can you show it ?

tonyhallett commented 1 year ago

Note that the error has come from a fork of ReportGenerator. FCC cannot use ReportGenerator beyond a certain version. The fork allows for some later commits to be included.

bb-avi commented 1 year ago

The file exists. I have scrubbed its contents and shared it privately with the email address listed on new issues.

I have dotnet-reportgenerator-globaltool v5.1.21 installed. Should I be using a different version of reportgenerator?

tonyhallett commented 1 year ago

We use a fork of report generator.

You need to share the report with me, email address to follow. It will probably be necessary to see the generated cobertura when Unit Tests and Integration Tests are run alone. Ideally if you were able to create a new minimal solution that exhibited the same behaviour then I could debug the fork and understand why it occurred.
I believe that the fork is this branch https://github.com/tonyhallett/ReportGeneratorForkFCC/tree/fcc-ms-generic-class-names - you could debug it yourself for a quicker resolution.

tonyhallett commented 1 year ago

Only other suggestion is you temporarily uninstall dotnet-reportgenerator-globaltool v5.1.21

tonyhallett commented 1 year ago

Do you get the same issue if you do not supply your own runsettings ?

tonyhallett commented 1 year ago

FCC uses ReportGenerator twice - one to combine cobertura files into a single cobertura file and then to produce the html report. It is the first step that is showing the error.

For MS code coverage there is only a single cobertura file. I have run your scrubbed cobertura file through the ReportGenerator fork and there is no error and report generator does output a cobertura file that should produce the html report.

Either there is an issue with the unscrubbed or the cobertura report is not present when the file path is passed to ReportGenerator.

FCC uses the Test Window API to get the file

namespace Microsoft.VisualStudio.TestWindow.Extensibility
{
    public interface IOperation
    {
        TestOperationStates Kind { get; }

        TimeSpan TotalRuntime { get; }

        IEnumerable<Uri> GetRunSettingsDataCollectorResultUri(Uri collectorUri);
    }
}
---
public const string MsDataCollectorUri = "datacollector://Microsoft/CodeCoverage/2.0";
--
        public static IEnumerable<Uri> GetRunSettingsMsDataCollectorResultUri(this IOperation operation)
        {
            return operation.GetRunSettingsDataCollectorResultUri(new Uri(RunSettingsHelper.MsDataCollectorUri));
        }
--
        private string[] GetCoberturaFiles(IOperation operation)
        {
            var resultsUris = operation.GetRunSettingsMsDataCollectorResultUri();
            var coberturaFiles = new string[0];
            if (resultsUris != null)
            {
                coberturaFiles = resultsUris.Select(uri => uri.LocalPath).Where(f => f.EndsWith(".cobertura.xml")).ToArray();
            }
            return coberturaFiles;
        }

Can you have the FCC log window open at the same time as the output folder to see if there is a cobertura file present C:\company\git\dotnet\Packages[REDACTED]\tests[REDACTED].UnitTests\bin\Debug\net7.0\fine-code-coverage\coverage-tool-output\c50ed9ad-c418-4dac-92b4-e2391ba106ce

tonyhallett commented 1 year ago

Apologies I debugged on the wrong branch - now have an error.

tonyhallett commented 1 year ago

Please update the extension - should be ok now.

bb-avi commented 1 year ago

Fix Confirmed! I didn't need to uninstall dotnet-reportgenerator-globaltool or remove my *.runsettings file.

Thank you @tonyhallett !!