search

dotnet / roslyn-sdk

Roslyn-SDK templates and Syntax Visualizer
MIT License
517 stars 256 forks source link

AnalyzerTesting: Make the generated syntax trees of source generators available #1074

Open tom-englert opened 1 year ago

tom-englert commented 1 year ago

The current approach to test source generators by providing the expected generated code upfront is not very convenient, since the generated sources can be very large, making the test unreadable.

It would be great to support another scenario, where the analyzer test is run with TestBehaviors.SkipGeneratedSourcesCheck, and then the generated sources can be fetched and verified with custom logic, e.g. using https://github.com/VerifyTests/ like in this sample https://github.com/Fody/PropertyChanged/blob/dee9d81bab0202dc2e5d985910072ad6e6c94fa4/PropertyChanged.Fody.Analyzer.Tests/CodeGeneratorTest.cs#L55-L71

sharwell commented 1 year ago

I believe microsoft/vs-extension-testing offers what you are looking for: https://github.com/microsoft/vs-extension-testing/blob/b78ac2a63a471cc0a56b45af0a1d7b2473288202/src/Microsoft.VisualStudio.Extensibility.Testing.SourceGenerator.UnitTests/Verifiers/CSharpSourceGeneratorVerifier%601%2BTest.cs#L4-L9

https://github.com/microsoft/vs-extension-testing/blob/b78ac2a63a471cc0a56b45af0a1d7b2473288202/src/Microsoft.VisualStudio.Extensibility.Testing.SourceGenerator.UnitTests/TestServicesSourceGeneratorTests.cs#L181-L202

tom-englert commented 1 year ago

Yes, you already can do many things if you dig into the details and rewrite the test framework, but it's very annoying that you need to do this again and again for every test - see #1080

tom-englert commented 1 year ago

I believe microsoft/vs-extension-testing offers what you are looking for

That example mimics the very basic approach, but not with the full power and comfort of VerifyTests

Also this sample rewrites a big part of the test, and of course depends on the test verifier, so it has to be rewritten for every verifier.

I already know how to do it with a custom written test, I was looking for a solution that runs out of the box, with the default test framework.

sharwell commented 1 year ago

This is similar to #1087, in that I'm not planning to directly implement it in the short term, but hopefully the future API to test the incrementality of source generators will allow for more direct validation of syntax trees produced by the source generators.

tom-englert commented 1 year ago

Actually as mentioned in the already closed PR #1075 I have a good enough solution here: https://github.com/tom-englert/RoslynAnalyzerTesting.CSharp.Extensions/blob/master/source/AnalyzerTesting.CSharp.Extensions/CSharpIncrementalGeneratorSnapshotTest.cs