coverlet-coverage / coverlet

Cross platform code coverage for .NET
MIT License
2.94k stars 385 forks source link

Coverlet misreports inline functions coverage #1225

Open AlexeyRaga opened 2 years ago

AlexeyRaga commented 2 years ago

It looks like Coverlet misses the inline directive on F# functions and reports zero coverage for such functions and methods:

image

In this example above, many of the functions that I test use unwrapHandler but, zero coverage is returned.

Another example: image

Here we see that valueCurry is used within the same method, but it is still reported as untested.

If I remove inline then it looks like:

image

So, even in my simple case, Coverlet misses 6K+ passes and reports the code as untested.

daveMueller commented 2 years ago

Thanks for reporting.

daveMueller commented 2 years ago

OK this is really complicated. I looked into the IL and saw that the compiler is generating a second function for inline functions with the exact same sequence point but with $W suffixed. This is described here https://github.com/fsharp/fslang-design/blob/main/FSharp-5.0/FS-1071-witness-passing-quotations.md#compiled-form-of-srtp-constrained-generic-code.

Coverlet is properly injecting the tracker in both functions but nevertheless they both never get called. Or at least for me it seems like this.

When I look into the IL without inline there is only one function without the $W. The function is also properly instrumented and the function is beeing call and thus hit.

image

Now when I look into the same IL with inline none of the two instrumented functions is called. Seems like the compiler optimized it in a way that the corresponding IL operation is used instead of a function call.

image

Maybe this is just because of the simple code example I'm using but I try to analyze this a bit more. Any information on this is highly welcome.

AlexeyRaga commented 2 years ago

Coverlet is properly injecting the tracker in both functions but nevertheless they both never get called.

I think that this is what inline means: the compiler inlines (copies) the function body instead of the function call.

From F# documentation:

Inline functions are functions that are integrated directly into the calling code.

I also see that inline in F# works across compilation units:

F# can also inline between compiled assemblies because inline is conveyed via .NET metadata.

So based on that I would expect that there'd be no calls to the inline functions and that they are only compiled into assemblies as functions (with the flag mentioned above) to be inlined later in assemblies that use them...

But I don't know what happens at the edge cases, like:

daveMueller commented 2 years ago

Thanks for the explanation. I currently don't have any idea how we could work around this. I can't find a way to map the inline function body to the executed IL. The only thing that comes to my mind is excluding inline functions from the instrumentation to at least get rid of the false not covered report of inline functions. @MarcoRossignoli @petli what do you think? Any other ideas?

daveMueller commented 2 years ago

OK I just got a response from the fsharp team that they currently don't emit this information (https://github.com/dotnet/fsharp/issues/12263). Now I really don't have any other idea how to deal with this except for excluding inline functions.