Issues with Fine Code Coverage Functionality

[IharSlaunikau]

Installed product versions

• Visual Studio: 2022
• This extension: 1.1.214

Description

Until recently, it had operated without any issues, however, it suddenly ceased providing the expected functionalities. The tool had been correctly tracking and reporting code coverage, providing valuable insights into my coding activity and efficiency. Lately, it has stopped working as expected, despite no changes made to the settings or the overall coding environment from my side, which has already significantly impacted my work progress and performance.

The precise issue seems to be unclear at this point. I have attempted basic troubleshooting steps such as rebooting the program and ensuring I am running the most current version of the software, all to no avail.

[tonyhallett]

You need to provide more to go on. It is probable that you are not including the SUT. What are your settings ? What is the FCC output ? Do you have multiple installs of Visual Studio ?

[bjornhellander]

I have the same experience. For example in this repo: https://github.com/DotNetAnalyzers/StyleCopAnalyzers Latest versions of Visual Studio and FCC. Unclear if it stopped working after a VS update, FCC update, etc. It works when having RunMsCodeCoverage=No, but that is unbearable in repositories of this size. I have 2022 and 2019 installed, but only using 2022.

FCC output when running a single test class, in case it helps:

Fine Code Coverage 4/12/2024 7:48:44 AM: Initializing
Fine Code Coverage 4/12/2024 7:48:47 AM: Initialized
Fine Code Coverage 4/12/2024 7:50:58 AM: Ms code coverage
Fine Code Coverage 4/12/2024 7:51:16 AM: ReportGenerator Run Arguments [reporttype:Cobertura] 
"-targetdir:C:\Users\bjorn\source\repos\StyleCopAnalyzers\StyleCop.Analyzers\StyleCop.Analyzers.Test\bin\Debug\net452\fine-code-coverage\coverage-tool-output"
"-reports:C:\Users\bjorn\source\repos\StyleCopAnalyzers\StyleCop.Analyzers\StyleCop.Analyzers.Test\bin\Debug\net452\fine-code-coverage\coverage-tool-output\c78ece0f-e9f0-4438-a64d-ec5959aa0be9\bjorn_LAPTOP-42S1MLQC_2024-04-12.07_51_13.cobertura.xml"
"-reporttypes:Cobertura"
Fine Code Coverage 4/12/2024 7:51:18 AM: ReportGenerator Run [reporttype:Cobertura]
2024-04-12T07:51:16: Arguments
2024-04-12T07:51:16:  -targetdir:C:\Users\bjorn\source\repos\StyleCopAnalyzers\StyleCop.Analyzers\StyleCop.Analyzers.Test\bin\Debug\net452\fine-code-coverage\coverage-tool-output
2024-04-12T07:51:16:  -reports:C:\Users\bjorn\source\repos\StyleCopAnalyzers\StyleCop.Analyzers\StyleCop.Analyzers.Test\bin\Debug\net452\fine-code-coverage\coverage-tool-output\c78ece0f-e9f0-4438-a64d-ec5959aa0be9\bjorn_LAPTOP-42S1MLQC_2024-04-12.07_51_13.cobertura.xml
2024-04-12T07:51:16:  -reporttypes:Cobertura
2024-04-12T07:51:18: Writing report file 'C:\Users\bjorn\source\repos\StyleCopAnalyzers\StyleCop.Analyzers\StyleCop.Analyzers.Test\bin\Debug\net452\fine-code-coverage\coverage-tool-output\Cobertura.xml'
2024-04-12T07:51:18: Report generation took 1.5 seconds
Fine Code Coverage 4/12/2024 7:51:18 AM: ReportGenerator Run Arguments [reporttype:HtmlInline_AzurePipelines] 
"-targetdir:C:\Users\bjorn\source\repos\StyleCopAnalyzers\StyleCop.Analyzers\StyleCop.Analyzers.Test\bin\Debug\net452\fine-code-coverage\coverage-tool-output"
"-reports:C:\Users\bjorn\source\repos\StyleCopAnalyzers\StyleCop.Analyzers\StyleCop.Analyzers.Test\bin\Debug\net452\fine-code-coverage\coverage-tool-output\Cobertura.xml"
"-plugins:c:\users\bjorn\appdata\local\microsoft\visualstudio\17.0_4f989535\extensions\snjomzfv.xqe\ReportGeneratorPlugins.dll"
"-reporttypes:FccLight"
"riskHotspotsAnalysisThresholds:metricThresholdForCyclomaticComplexity=30"
"riskHotspotsAnalysisThresholds:metricThresholdForCrapScore=15"
"riskHotspotsAnalysisThresholds:metricThresholdForNPathComplexity=200"
Fine Code Coverage 4/12/2024 7:51:19 AM: ReportGenerator Run [reporttype:HtmlInline_AzurePipelines]
2024-04-12T07:51:18: Arguments
2024-04-12T07:51:18:  -targetdir:C:\Users\bjorn\source\repos\StyleCopAnalyzers\StyleCop.Analyzers\StyleCop.Analyzers.Test\bin\Debug\net452\fine-code-coverage\coverage-tool-output
2024-04-12T07:51:18:  -reports:C:\Users\bjorn\source\repos\StyleCopAnalyzers\StyleCop.Analyzers\StyleCop.Analyzers.Test\bin\Debug\net452\fine-code-coverage\coverage-tool-output\Cobertura.xml
2024-04-12T07:51:18:  -plugins:c:\users\bjorn\appdata\local\microsoft\visualstudio\17.0_4f989535\extensions\snjomzfv.xqe\ReportGeneratorPlugins.dll
2024-04-12T07:51:18:  -reporttypes:FccLight
2024-04-12T07:51:18:  riskHotspotsAnalysisThresholds:metricThresholdForCyclomaticComplexity=30
2024-04-12T07:51:18:  riskHotspotsAnalysisThresholds:metricThresholdForCrapScore=15
2024-04-12T07:51:18:  riskHotspotsAnalysisThresholds:metricThresholdForNPathComplexity=200
2024-04-12T07:51:18: Writing report file 'C:\Users\bjorn\source\repos\StyleCopAnalyzers\StyleCop.Analyzers\StyleCop.Analyzers.Test\bin\Debug\net452\fine-code-coverage\coverage-tool-output\index.html'
2024-04-12T07:51:19: Report generation took 0.6 seconds
Fine Code Coverage 4/12/2024 7:51:19 AM: Processing cobertura
Fine Code Coverage 4/12/2024 7:51:19 AM: Processing report
Fine Code Coverage 4/12/2024 7:51:20 AM: ================================== DONE ==================================

Options:

[tonyhallett]

Please set IncludeReferencedProjects to true

[tonyhallett]

. I have 2022 and 2019 installed, but only using 2022.

If FCC is installed in both then the version of FCC needs to be the same.

@bjornhellander

[bjornhellander]

@tonyhallett Ok. I checked before commenting and it is only installed in 2022, but good to know 👍 IncludeReferencedProjects = true seems very reasonable. :-) I will try later today, but I have no recollection of changing anything except RunMsCodeCoverage. Has something changed in FCC, or do you think I have accidentally toggled this setting?

[tonyhallett]

@bjornhellander I think I corrected IncludeTestAssembly for ms code coverage which may then require the explicit inclusion of the SUT.

[IharSlaunikau]

[IharSlaunikau]

only VS 2022

[tonyhallett]

@IharSlaunikau Looks like you have disabled FCC

[bjornhellander]

@tonyhallett Worked like a charm! Thanks! @IharSlaunikau Sorry for hi-jacking your issue! I thought I was experiencing the same problem as you.

[tonyhallett]

@bjornhellander Excellent. 1 down 1 to go !

[senox13]

I believe I might be having the same issue here. Been using FCC for years with no issues, but all of a sudden it's just flat-out stopped functioning. No color-coding in the margin, no report generated, no fine-code-coverage-net6.0 folder generated under bin/Debug, nothing at all to suggest what the issue is.

VS 2022 17.7.2 FCC 1.1.214

I emphasize that I might be having the same issue because unlike @IharSlaunikau, I see only the following in the FCC section of the output window after running any code.

Fine Code Coverage 4/20/2024 2:19:31 PM: Initializing
Fine Code Coverage 4/20/2024 2:19:31 PM: Initialized

I've verified that the settings ShowEditorCoverage, IncludeReferencedProjects, IncludeTestAssembly, and Enabled, are all set to True, and I've tried running with RunMsCodeCoverage set to both Yes and No. I'm happy to provide a demo solution if it'd be helpful, but I've been troubleshooting this using a completely barebones hello world project to no avail, so it definitely seems to be some issue with my broader IDE setup rather than any specific project.

If this seems like an unrelated issue, please let me know and I'll be happy to open up my own separate issue. Otherwise, please let me know if there's any other info I can provide. I'm sorely missing this incredibly useful tool.

[tonyhallett]

@senox13 Do you have multiple Visual Studio installations ?

What are the versions of the utilities in here ?

[senox13]

Apologies for the delay, I've had a busy couple days and am just now getting back to things. I do have multiple versions of VS installed, 2022 and 2019.

Within AppData\Local\FineCodeCoverage, I have these subfolders containing these versions: coverlet: 3.2.0 coverletCollector: 3.2.0 msCodeCoverage: 17.9.0 msTestPlatform: 16.9.1 openCover: 4.7.922 reportGenerator: 4.7.4

Thank you

[tonyhallett]

@senox13 Do you have the same version of FCC in both VS versions ?

[senox13]

I don't have FCC installed for VS 2019, in fact I haven't opened it in very likely well over a year. If you think it'd make a difference, I'll go ahead and uninstall it right now

[tonyhallett]

@senox13 No no need, just checking.

Please can you provide your hello world solution.

[senox13]

CoverageTesting.zip

Absolutely, here you are. I just tested it again, in case things magically fixed themselves over the weekend, but I'm seeing exactly the same behaviour that I initially described.

Thank you so much for taking the time to look into this, I really appreciate the fast responses

[tonyhallett]

That's never going to work. You a need a test project.

[senox13]

Oh! My mistake, I could have sworn I could get coverage information from a conventional run without tests. I've added a demo test project to that solution, but unfortunately, I still don't get any output in the Coverage tab of the Fine Code Coverage pane, or color-coding in the margins. The FCC output window does however suggest that a report is being generated. I've attached the modified solution in case I'm missing anything else obvious.

CoverageTesting.zip

[tonyhallett]

@senox13 What are you doing to collect coverage ?

[tonyhallett]

Are you using ms code coverage ? What options have you set ? Have you set IncludeReferencedProjects ?

[senox13]

Hello,

I have attached screenshots of all of my FCC settings. I've tried running with RunMsCodeCoverage set to both Yes and No, with no difference in the results. At this point, I am getting the below table in the FCC section of the output window, so coverage is definitely being measured, but the coverage tab of the FCC pane remains completely blank.

+-----------------+------+--------+--------+
| Module          | Line | Branch | Method |
+-----------------+------+--------+--------+
| CoverageTesting | 50%  | 100%   | 50%    |
+-----------------+------+--------+--------+

+---------+------+--------+--------+
|         | Line | Branch | Method |
+---------+------+--------+--------+
| Total   | 50%  | 100%   | 50%    |
+---------+------+--------+--------+
| Average | 50%  | 100%   | 50%    |
+---------+------+--------+--------+

[tonyhallett]

@senox13

What is the full directory path to the CoverageTesting.csproj ?

[senox13]

Nothing egregiously long or out of the ordinary: C:\Users\localuser\Desktop\Code\__working__\CoverageTesting\CoverageTesting\CoverageTesting.csproj

[tonyhallett]

Can you update visual studio ?

[tonyhallett]

@senox13

I am getting the results with both coverage providers.

Do you have the Fine Code Coverage window open when you run tests ?

Can you show the full output of the FCC output window pane.

Can you clear this directory C:\Users\tonyh\Downloads\CoverageTesting\CoverageTesting\Tests\bin\Debug\net6.0\fine-code-coverage\coverage-tool-output

and run once with ms code coverage and once without. With each run can you see if an index-processed.html file is generated and if so what happens when you open it. With ms code coverage there should be Tests-fcc-mscodecoverage-generated.runsettings. Can you post the contents of this file.

[senox13]

Alright, I updated VS 2022, uninstalled VS 2019, and reinstalled FCC. It appears now that everything is working as expected. Thank you so much for bearing with me on this, now I can get back to writing tests with some confidence that I'm actually covering all of my projects' code. Two down, one to go!