search

bazelbuild / rules_dotnet

.NET rules for Bazel
Apache License 2.0
187 stars 82 forks source link

Support for coverage testing #359

Open Inirit opened 1 year ago

Inirit commented 1 year ago

I'm working on a Bazel-based project that has a goal of enabling code coverage unit testing for our supported languages, one of those languages being C#. However, if I understand correctly, rules_dotnet does not currently support code coverage which puts us in a bind. After spending quite a bit of time investigating how to enable this support without requiring upstream changes to rules_dotnet, we've come to the conclusion that this is likely just not possible (at least not without some very significant compromises and hacky workarounds).

I can go into far more detail of what we're trying to do and how we've tried to do it (I've almost got something working locally with a handful of hacks and a private fork of rules_dotnet), but first I'd like to get initial impressions. Does code coverage support sound possible? Are there plans for it? Any ideas for how it could be done?

purkhusid commented 1 year ago

I'll be honest an just say that I have never added coverage support for anything in Bazel before so I don't really have any prior experience to lean on. If you can find a good way to add coverage support then I would gladly accept the contribution.

Inirit commented 1 year ago

Got it, I appreciate the honesty. I'll discuss possible directions with my team.

While that's happening, there's at least two required changes that I'd like to point out for the sake of possible discussion:

NuGet package "build" folder

The NuGet package folder structure convention documented by Microsoft lists a folder called "build" that lives alongside other folders like "lib" and "ref". According to the documentation, this folder is meant to contain .props and .targets files, i.e. MSBuild files. Currently rules_dotnet does not include this "build" folder in its package processing and on the surface this makes sense.

However, there are some NuGet packages that include more than just .props/.targets files in the "build" folder, e.g. NUnit3TestAdapter and coverlet.collector. These packages include additional assemblies that are meant to be referenced not during run time but instead during build/test time. Bazel needs to be made aware of these assemblies so that they can be leveraged, otherwise rules_dotnet interprets these packages as essentially empty.

Transitive PDBs

rules_dotnet does not currently include PDBs as transitive dependencies. This results in the build output of a test project lacking the PDB of the actual assembly being tested. This becomes a tangible problem when using coverlet for coverage testing support because it only instruments assemblies that have corresponding PDBs.