Open ryanbrandenburg opened 5 years ago
I'm hoping this comes out of recent work to bring up Coverlet on dotnet/machinelearning and dotnet/coreclr. In particular, dotnet/machinelearning has figured out a nice combination of Coverlet+ReportGenerator to produce files consumable by codecov.io.
Any updates @sharwell ?
Not from me, but @ViktorHofer may have updates
Relates to https://github.com/dotnet/arcade/issues/2076. After enabling full code coverage in corefx (with codecov) I can help out here.
Thanks @ViktorHofer
Hi @ViktorHofer, I see that this issue is a bit old, so checking into see if any requirements have changed on this? Thanks!
@pranavkm seems to have a draft PR for doing something like this over at #3919.
VSTest now supports all the feature that we need. I recommend to use dotnet test's coverlet integration instead of invoking coverlet directly. The necessary work here would be:
dotnet test --collect:"XPlat Code Coverage"
(https://github.com/tonerdo/coverlet#vstest-integration-preferred-due-to-known-issue-supports-only-net-core-application)Optional:
I'm currently not actively working on these things...
@ViktorHofer Thanks for the update! I've implemented something similar to your suggestion in our services' CI for code coverage, too, so we'll see how much of that we can leverage for this addition to Arcade.
Happy to help providing something standard using Microsoft Code Coverage
if we are interested. I have tried to create a PR on roslyn-analyzers and it seems that the changes required for coverage only is really small.
Adding
<TestRunnerAdditionalArguments>--collect:"Code Coverage;Format=Cobertura"</TestRunnerAdditionalArguments>
<!-- TODO: On next arcade bump of Microsoft.CodeCoverage/Microsoft.NET.Test.Sdk we will have to update this line to be netstandard2.0 -->
<!-- Because arcade is passing dlls directly to 'dotnet test' command, we need to manually provide path to coverage datacollector -->
<TestRunnerAdditionalArguments>$(TestRunnerAdditionalArguments) --test-adapter-path $(NUGET_PACKAGES)/microsoft.codecoverage/$(MicrosoftNETTestSdkVersion)/build/netstandard1.0</TestRunnerAdditionalArguments>
in some props/target file seems to be enough. Obviously, it would be better to also provide a .runsettings
.
It would be nice to add targets/properties to allow code coverage report generation out of the box in Arcade. Bonus points if it produces a format that AZDO pipelines can ingest by default.
After fiddling with things a bit it seems that the
dotnet exec $(TestRunnerPath)
method of executing tests used by the Xunit target prevents using packages likecoverlet.msbuild
, so an implementation in Arcade or documentation of a simple way to DIY would be ideal.