search

microsoft / codecoverage

MIT License
84 stars 11 forks source link

Support OpenCover output format #98

Open josundt opened 9 months ago

josundt commented 9 months ago

OpenCover is the best format when integrating with SonarCloud/SonarQube. Sonar supports MS VS .coverage (binary) and VS coverage xml formats as well, but these formats give unexpected line coverage metrics (see #97).

Until OpenCover is a supported output format, coverlet remains the preferred option for us,

josundt commented 9 months ago

I'll give you some more details about my findings:

  1. In the Azure DevOps pipeline after switching from Cobertura (using Publish Code Coverage Results v1 task) to MS Coverage XML (using Publish Code Coverage Results v2 task), the reported "lines covered" remained exactly the same 👍

  2. With SonarCloud after switching from OpenCover to MS Coverage XML, the "lines covered" count dropped by 50%.

I looks like the main problem may be SonarCloud rather than the XML file written by your tool. Still the only currently acceptable solution for us is to revert back to OpenCover and coverlet.

If your tooling had supported the OpenCover output format (and ideally also an improved way specifying output file or folder paths when using multiple files in different formats), we would have started to use Microsoft.CodeCoverage instead.

SeMuell commented 8 months ago

After doing some analysis I totally agree that supporting OpenCover would be very beneficial. We have to stick to SonarQube and the analysis takes around 3-5 times as long with the VS coverage xml format than with OpenCover produced by coverlet.

Unfortunately, all performance benefits we gain using the Microsoft.CodeCoverage in comparison to coverlet is taken by not supporting OpenCover.

Until OpenCover is a supported output format, coverlet remains the preferred option for us as well.

martin-strecker-sonarsource commented 5 months ago

Hello @josundt

The issue you are seeing might be related to https://github.com/SonarSource/sonar-dotnet/issues/3191. Can you provide us a reproducer? We are looking for code patterns that are not properly imported by our coverage importer. One example is methods using yield return, but there might be more such patterns.