The AnalyzerTest<TVerifier>.MatchDiagnostics is implemented recursively with a local function "RecursiveMatch", resulting in an access violation of the .NET 8.0 RC runtime during a stack overflow when testing my source generator on moderately sized bad inputs. With some effort I was able to capture a full dump of the runtime crash I can share privately if required, but as far as this repo is concerned the analysis of the issue seems clear:
full stack trace
```
Microsoft.CodeAnalysis.Analyzer.Testing.dll!Microsoft.CodeAnalysis.Testing.AnalyzerTest.MatchDiagnostics.__RecursiveMatch|70_6(int firstActualIndex, int remainingActualItems, int firstExpectedIndex, int remainingExpectedItems, Microsoft.CodeAnalysis.Testing.MatchQuality unmatchedActualResults, bool[] usedExpected, ref Microsoft.CodeAnalysis.Testing.AnalyzerTest.<>c__DisplayClass70_0 value) Line 491 C#
Microsoft.CodeAnalysis.Analyzer.Testing.dll!Microsoft.CodeAnalysis.Testing.AnalyzerTest.MatchDiagnostics.__RecursiveMatch|70_6(int firstActualIndex, int remainingActualItems, int firstExpectedIndex, int remainingExpectedItems, Microsoft.CodeAnalysis.Testing.MatchQuality unmatchedActualResults, bool[] usedExpected, ref Microsoft.CodeAnalysis.Testing.AnalyzerTest.<>c__DisplayClass70_0 value) Line 573 C#
[The 1 frame(s) above this were repeated 665 times]
Microsoft.CodeAnalysis.Analyzer.Testing.dll!Microsoft.CodeAnalysis.Testing.AnalyzerTest.MatchDiagnostics.__RecursiveMatch|70_6(int firstActualIndex, int remainingActualItems, int firstExpectedIndex, int remainingExpectedItems, Microsoft.CodeAnalysis.Testing.MatchQuality unmatchedActualResults, bool[] usedExpected, ref Microsoft.CodeAnalysis.Testing.AnalyzerTest.<>c__DisplayClass70_0 value) Line 573 C#
Microsoft.CodeAnalysis.Analyzer.Testing.dll!Microsoft.CodeAnalysis.Testing.AnalyzerTest.MatchDiagnostics((Microsoft.CodeAnalysis.Project project, Microsoft.CodeAnalysis.Diagnostic diagnostic)[] actualResults, Microsoft.CodeAnalysis.Testing.DiagnosticResult[] expectedResults) Line 474 C#
Microsoft.CodeAnalysis.Analyzer.Testing.dll!Microsoft.CodeAnalysis.Testing.AnalyzerTest.VerifyDiagnosticResults(System.Collections.Generic.IEnumerable<(Microsoft.CodeAnalysis.Project project, Microsoft.CodeAnalysis.Diagnostic diagnostic)> actualResults, System.Collections.Immutable.ImmutableArray analyzers, Microsoft.CodeAnalysis.Testing.DiagnosticResult[] expectedResults, Microsoft.CodeAnalysis.Testing.IVerifier verifier) Line 348 C#
Microsoft.CodeAnalysis.Analyzer.Testing.dll!Microsoft.CodeAnalysis.Testing.AnalyzerTest.VerifyDiagnosticsAsync(Microsoft.CodeAnalysis.Testing.Model.EvaluatedProjectState primaryProject, System.Collections.Immutable.ImmutableArray additionalProjects, Microsoft.CodeAnalysis.Testing.DiagnosticResult[] expected, Microsoft.CodeAnalysis.Testing.IVerifier verifier, System.Threading.CancellationToken cancellationToken) Line 273 C#
[Resuming Async Method]
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(System.Threading.Thread threadPoolThread, System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 264 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder.AsyncStateMachineBox.d__66>.MoveNext(System.Threading.Thread threadPoolThread) Line 376 C#
System.Private.CoreLib.dll!System.Threading.ThreadPoolWorkQueue.Dispatch() Line 913 C#
System.Private.CoreLib.dll!System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart() Line 58 C#
[Native to Managed Transition]
kernel32.dll!00007ffb09d37344() Unknown
ntdll.dll!00007ffb0acc26b1() Unknown
[Async Call Stack]
[Async] Microsoft.CodeAnalysis.SourceGenerators.Testing.dll!Microsoft.CodeAnalysis.Testing.SourceGeneratorTest.RunImplAsync(System.Threading.CancellationToken cancellationToken) Line 45 C#
[Async] CodeGeneratorTests.dll!CodeGeneratorTests.CSharpSourceGeneratorVerifier.Test.RunImplAsync(System.Threading.CancellationToken ct) Unknown
[Async] Microsoft.CodeAnalysis.Analyzer.Testing.dll!Microsoft.CodeAnalysis.Testing.AnalyzerTest.RunAsync(System.Threading.CancellationToken cancellationToken) Line 181 C#
[Async] CodeGeneratorTests.dll!CodeGeneratorTests.WindowsInteropTest.GenerateEquatable() Unknown
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.TestInvoker.InvokeTestMethodAsync.AnonymousMethod__1() Line 264 C#
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.ExecutionTimer.AggregateAsync(System.Func asyncAction) Line 48 C#
[Async] xunit.core.dll!Xunit.Sdk.ExceptionAggregator.RunAsync(System.Func code) Line 90 C#
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.TestInvoker.InvokeTestMethodAsync(object testClassInstance) Line 241 C#
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.TestInvoker.RunAsync.AnonymousMethod__47_0() Line 206 C#
[Async] xunit.core.dll!Xunit.Sdk.ExceptionAggregator.RunAsync(System.Func> code) Line 107 C#
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.XunitTestRunner.InvokeTestAsync(Xunit.Sdk.ExceptionAggregator aggregator) Line 67 C#
[Async] xunit.core.dll!Xunit.Sdk.ExceptionAggregator.RunAsync>(System.Func>> code) Line 107 C#
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.TestRunner.RunAsync() Line 149 C#
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.TestCaseRunner.RunAsync() Line 82 C#
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.TestMethodRunner.RunTestCasesAsync() Line 136 C#
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.TestMethodRunner.RunAsync() Line 106 C#
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.TestClassRunner.RunTestMethodsAsync() Line 213 C#
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.TestClassRunner.RunAsync() Line 171 C#
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.TestCollectionRunner.RunTestClassesAsync() Line 130 C#
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.TestCollectionRunner.RunAsync() Line 101 C#
[Async] System.Private.CoreLib.dll!System.Threading.Tasks.Task.Run C#
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.XunitTestAssemblyRunner.RunTestCollectionsAsync(Xunit.Sdk.IMessageBus messageBus, System.Threading.CancellationTokenSource cancellationTokenSource) Line 205 C#
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.TestAssemblyRunner.RunAsync() Line 196 C#
[Async] xunit.execution.dotnet.dll!Xunit.Sdk.XunitTestFrameworkExecutor.RunTestCases(System.Collections.Generic.IEnumerable testCases, Xunit.Abstractions.IMessageSink executionMessageSink, Xunit.Abstractions.ITestFrameworkExecutionOptions executionOptions) Line 95 C#
```
Note that the stack trace has ~670 frames before crashing the runtime, looking at the variables on the stack there are ~800 items to compare and the recursion decrements by one, so my guess is that it would have eventually completed if it weren't triggering an early stack overflow (which the dotnet runtime apparently wasn't prepared to handle and panics).
The fact that the test infrastructure crashes the runtime means the tests don't light up as failures nor can you attach a debugger and look for what is crashing (the debugger cannot catch the runtime failure either), so you are left with an arbitrary number of tests just not having been run.
The
AnalyzerTest<TVerifier>.MatchDiagnosticsis implemented recursively with a local function "RecursiveMatch", resulting in an access violation of the .NET 8.0 RC runtime during a stack overflow when testing my source generator on moderately sized bad inputs. With some effort I was able to capture a full dump of the runtime crash I can share privately if required, but as far as this repo is concerned the analysis of the issue seems clear:full stack trace
``` Microsoft.CodeAnalysis.Analyzer.Testing.dll!Microsoft.CodeAnalysis.Testing.AnalyzerTestNote that the stack trace has ~670 frames before crashing the runtime, looking at the variables on the stack there are ~800 items to compare and the recursion decrements by one, so my guess is that it would have eventually completed if it weren't triggering an early stack overflow (which the dotnet runtime apparently wasn't prepared to handle and panics).
The fact that the test infrastructure crashes the runtime means the tests don't light up as failures nor can you attach a debugger and look for what is crashing (the debugger cannot catch the runtime failure either), so you are left with an arbitrary number of tests just not having been run.