search

ChilliCream / graphql-platform

Welcome to the home of the Hot Chocolate GraphQL server for .NET, the Strawberry Shake GraphQL client for .NET and Banana Cake Pop the awesome Monaco based GraphQL IDE.
https://chillicream.com
MIT License
5.26k stars 746 forks source link

StrawberryShake generator messes with Coverlet #3811

Closed nover closed 3 years ago

nover commented 3 years ago

Describe the bug When a project has the StrawberryShake code generator added as a reference, the entire project will disappear from coverage reports when using coverlet console or msbuild.

I have created an issue with coverlet here and they are hinting at the fact that it might be because the code-generator does not obey the "convention" that generated code should end wit h*.g.cs.

To Reproduce Steps to reproduce the behavior:

  1. Setup a test suite with coverlet.msbuild
  2. Run some tests where `StrawberryShake.CodeGeneration.CSharp.Analyzers is not added
  3. Notice that your code is covered
  4. Add StrawberryShake.CodeGeneration.CSharp.Analyzers and see your test coverage disappear

Expected behavior That code coverage works even when StrawberryShake.CodeGeneration.CSharp.Analyzers is added to my csproj file

So I'm raising this issue in the hope that StrawberryShake.CodeGeneration.CSharp.Analyzers can start following this "convention" of suffixing generated code files.

michaelstaib commented 3 years ago

The generated code is only for IDEs, we internally removed it from the compilation with <Compile Remove="" />.

Most tools that we have looked at have a problem with source generators.

michaelstaib commented 3 years ago

Looking at your steps to reproduce this is exactly the issue. There are several issues open at the roslyn repo with tooling. If you want to report the issue back to coverlet you can refer to this roslyn issue.

https://github.com/dotnet/roslyn/issues/51773

nover commented 3 years ago

Thanks for having a look @michaelstaib - seems like my hope for a quick fix has evaporated.

I'll see if we can move some code around to isolate the auto-generated stuff and retain coverage for the remainder of the code.

michaelstaib commented 3 years ago

There is a workaround for you!

What we internally at the moment do with sonar is to disable the analyzer on the build server. This works if you also checkin the generated code.

Simply put a condition on your package reference. 

<PackageReference Include="StrawberryShake.CodeGeneration.CSharp.Analyzers" Version="11.2.2" Condition="'$(DisableBerryAnalyzer)' == 'true'" />

you can pass in the property DisableBerryAnalyzer like the following on build... 

dotnet build -p: DisableBerryAnalyzer=false
michaelstaib commented 3 years ago

please close the issue if that helped you.

nover commented 3 years ago

@michaelstaib nice, I went with the approach of simply removing the package in our CI builds - the generated code is checked in, everything builds and our coverage is back.

Thank you 🎉

Sibusten commented 2 years ago

I ran into this issue and discovered a workaround that doesn't require committing the generated code. Thought I would share here in case someone else finds this.

In .graphqlrc.json, set the following:

"outputDirectoryName": "StrawberryShake.CodeGeneration.CSharp.Analyzers/StrawberryShake.CodeGeneration.CSharp.Analyzers.CSharpClientGenerator",
"emitGeneratedCode": true,

This causes the generated code to be emitted to the folder that Coverlet is looking for it in, restoring coverage support.