MS Coverage output isn't cleaning compiler generated methods/types

[erichiller]

Describe the bug When using Microsoft CodeCoverage (aka dotnet-coverage) which can now export in Cobertura format, the Cobertura is not cleaned up for compiler generated types and methods where the names are mangled.

A real example, for a single class, the following classes are present in the CodeCoverage output Cobertura file:

<class line-rate="0.7832167832167832" branch-rate="0.75" complexity="58" name="RequestState.RequestStateManager&lt;TRequestMessage, TResponseMessage, TChannelOutput&gt;" filename="RequestState/RequestStateManager.cs">
<class line-rate="1" branch-rate="1" complexity="4" name="RequestState.RequestStateManager.&lt;&lt;GetChannelForRequestAsync&gt;g__addDbStorageAsync|14_0&gt;d&lt;TRequestMessage, TResponseMessage, TChannelOutput&gt;" filename="RequestState/RequestStateManager.cs">
<class line-rate="1" branch-rate="1" complexity="2" name="RequestState.RequestStateManager.&lt;&gt;c&lt;TRequestMessage, TResponseMessage, TChannelOutput&gt;" filename="RequestState/RequestStateManager.cs">
<class line-rate="1" branch-rate="1" complexity="1" name="RequestState.RequestStateManager.&lt;&gt;c__DisplayClass19_0&lt;TRequestMessage, TResponseMessage, TChannelOutput&gt;" filename="RequestState/RequestStateManager.cs">
<class line-rate="0.6153846153846154" branch-rate="0.5" complexity="4" name="RequestState.RequestStateManager.&lt;ClearRequestStateAsync&gt;d__17&lt;TRequestMessage, TResponseMessage, TChannelOutput&gt;" filename="RequestState/RequestStateManager.cs">
<class line-rate="0.84" branch-rate="0.8" complexity="10" name="RequestState.RequestStateManager.&lt;CompleteRequestAsync&gt;d__19&lt;TRequestMessage, TResponseMessage, TChannelOutput&gt;" filename="RequestState/RequestStateManager.cs">
<class line-rate="1" branch-rate="1" complexity="1" name="RequestState.RequestStateManager.&lt;DisposeAsync&gt;d__33&lt;TRequestMessage, TResponseMessage, TChannelOutput&gt;" filename="RequestState/RequestStateManager.cs">
<class line-rate="0.8888888888888888" branch-rate="0.9166666666666666" complexity="12" name="RequestState.RequestStateManager.&lt;DisposeAsyncCore&gt;d__34&lt;TRequestMessage, TResponseMessage, TChannelOutput&gt;" filename="RequestState/RequestStateManager.cs">
<class line-rate="1" branch-rate="1" complexity="4" name="RequestState.RequestStateManager.&lt;mkmrk-DataSource-Ibkr-RequestState-IMessageStateManager-MarkSentAsync&gt;d__30&lt;TRequestMessage, TResponseMessage, TChannelOutput&gt;" filename="RequestState/RequestStateManager.cs">

Which results in the following classes being reported:

Name	Line	Branch	Method
RequestState.RequestStateManager.<>c<TRequestMessage, TResponseMessage, TChannelOutput>	100%
RequestState.RequestStateManager<TRequestMessage, TRe sponseMessage, TChannelOutput>	77.3%	75%	85.7%
RequestState.RequestStateManager<TRequestMessage, TRe sponseMessage, TChannelOutput>	87.3%	85.2%	100%
RequestState.RequestStateManager<TRequestMessage, TRe sponseMessage>	65.3%	66.6%	100%

To Reproduce Any use of Microsoft CodeCoverage on any codebase with constructs such as async methods, local functions, etc. causes these same issues.

[danielpalme]

Probably Microsoft CodeCoverage uses a slightly different formatting that other tools (e.g. coverlet). I will look into this within the next days and try to handle this format as well.

[danielpalme]

My assumption is correct. The format is different:

dotnet-test Test.ClassWithLocalFunctions.MyNestedClass<T1, T2> Test.ClassWithLocalFunctions.MyNestedClass.<MyAsyncMethod>d4<T1, T2, T3> Test.ClassWithLocalFunctions.MyNestedClass.<>cDisplayClass4_0.<<MyAsyncMethod>g__MyAsyncLocalFunction|0>d<T1, T2, T3, T4>

coverlet Test.ClassWithLocalFunctions1/MyNestedClass1 Test.ClassWithLocalFunctions1/MyNestedClass1/<MyAsyncMethod>d41 Test.ClassWithLocalFunctions1/MyNestedClass`1/<>cDisplayClass4_01/<<MyAsyncMethod>g__MyAsyncLocalFunction|0>d1

Will have to adjust in ReportGenerator

[danielpalme]

I invested several hours, but I have not yet found a way filter out those classes and at the same time don't miss any relevant coverage results.

[cremor]

@erichiller Do you know if the PublishCodeCoverageResults@2 Azure DevOps pipeline task handles this correctly?

@danielpalme Maybe I'm missing something, but couldn't you just exclude/merge all types that have an invalid name? The compiler generates those invalid names to make sure they will never clash with the actually written code. In the given examples they all start with <.

[danielpalme]

@cremor A < is not enough to exclude a class. E.g. generic class names also contain <.

[cremor]

I meant type.Name.StartsWith('<').

[danielpalme]

I will have a look again as soon as possible. I'm pretty busy at the moment so it may take some time.

[danielpalme]

I just made some change to improve the handling of Cobertura files generated by "Microsoft Coverage".

In your example the coverage file contains the following class names:

RequestState.RequestStateManager<TRequestMessage, TResponseMessage, TChannelOutput> RequestState.RequestStateManager.<gaddDbStorageAsync|14_0>d<TRequestMessage, TResponseMessage, TChannelOutput> RequestState.RequestStateManager.<>c<TRequestMessage, TResponseMessage, TChannelOutput> RequestState.RequestStateManager.<>c__DisplayClass19_0<TRequestMessage, TResponseMessage, TChannelOutput> RequestState.RequestStateManager.d17<TRequestMessage, TResponseMessage, TChannelOutput> RequestState.RequestStateManager.d19<TRequestMessage, TResponseMessage, TChannelOutput> RequestState.RequestStateManager.d33<TRequestMessage, TResponseMessage, TChannelOutput> RequestState.RequestStateManager.d34<TRequestMessage, TResponseMessage, TChannelOutput> RequestState.RequestStateManager.d30<TRequestMessage, TResponseMessage, TChannelOutput>

With my change, only the following element will appear in the report:

RequestState.RequestStateManager<TRequestMessage, TResponseMessage, TChannelOutput>

I will do some more testing and probably publish a new release within the next days.

[danielpalme]

I just released version 5.3.6 with some improvements regarding Microsoft CodeCoverage. Please let me know if this works (better) for you.

[standsed]

@danielpalme in some cases (async + generics + static) classes are still reported multiple times in 5.3.6 Name properties under class tags

Repro.Class1.<Method3>d__2<T, TU>
Repro.Class1.<Method2>d__1<T>
Repro.Class1.<Method1>d__0

Complete xml file

More details in microsoft/codecoverage issue

[danielpalme]

I will have a look in the next days.

[standsed]

@danielpalme for nested classes also getting duplicated statistics in report https://github.com/standsed/CodeCoverageRepro/blob/main/Repro/Class2.cs Class2 includes statistics also for Class3 Coverage report via Coverlet collector

[danielpalme]

@standsed I answered here on the problem: https://github.com/microsoft/codecoverage/issues/124#issuecomment-2171511622

TLDR: I can't fix this without causing other issues. Coverlet uses / to separate namespace and nested class names. Microsoft CodeCoverage uses . and therefore there is no way to distinct between namespaces and class name.