Add a performance stress test for solution with large number of projects + analyzers

[mavasani]

See https://github.com/dotnet/roslyn/issues/10365.

Dump for the issue shared at \<%internal_share%>\public\mavasani\10365_OOM shows that bulk of in memory objects are from analyzer state tracking:

2fdf332c    31212      8609712 System.Collections.Generic.Dictionary`2+Entry[[System.String, mscorlib],[System.Collections.Immutable.ImmutableArray`1[[Microsoft.CodeAnalysis.CSharp.Symbol, Microsoft.CodeAnalysis.CSharp]], System.Collections.Immutable]][]
2c140f8c   477217     10065724 Microsoft.CodeAnalysis.CSharp.Symbol[]
291e87e4    55380     10432856 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[System.Object, mscorlib],[Microsoft.CodeAnalysis.Diagnostics.EngineV1.DiagnosticIncrementalAnalyzer+AnalysisData, Microsoft.CodeAnalysis.Features]][]
2c145270   208388     10836176 Microsoft.CodeAnalysis.CSharp.Symbols.Metadata.PE.PEMethodSymbol
30e0f278      593     11573748 JetBrains.ReSharper.Psi.XmlIndex.XmlDocIndex+IndexEntry[]
71fe2518    24720     14875913 System.Byte[]
136eb46c  1487732     17889732 Microsoft.CodeAnalysis.Diagnostics.DiagnosticData[]
71fde23c     1781     20423092 System.Char[]
29207120  1487729     35705496 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[System.Object, mscorlib],[Microsoft.CodeAnalysis.Diagnostics.EngineV1.DiagnosticIncrementalAnalyzer+AnalysisData, Microsoft.CodeAnalysis.Features]]
296e8e50   105793     56441296 Microsoft.CodeAnalysis.SyntaxReference[]
2900d44c  1487729     71410992 Microsoft.CodeAnalysis.Diagnostics.EngineV1.DiagnosticIncrementalAnalyzer+AnalysisData
008673b0   532224     86914912      Free
2c118460 13695300    164343600 Microsoft.CodeAnalysis.CSharp.NamespaceDeclarationSyntaxReference
71fdf680   355401    174412232 System.Int32[]
71fdd834  2042444    192389066 System.String
298da964      116    290348464 System.Collections.Generic.Dictionary`2+Entry[[Microsoft.CodeAnalysis.ISymbol, Microsoft.CodeAnalysis],[System.Collections.Generic.Dictionary`2[[System.Int32, mscorlib],[Microsoft.CodeAnalysis.Diagnostics.AnalysisState+DeclarationAnalyzerStateData, Microsoft.CodeAnalysis]], mscorlib]][]
298ddbec      120    295416288 System.Collections.Generic.Dictionary`2+Entry[[Microsoft.CodeAnalysis.Diagnostics.CompilationEvent, Microsoft.CodeAnalysis],[Microsoft.CodeAnalysis.Diagnostics.AnalysisState+AnalyzerStateData, Microsoft.CodeAnalysis]][]
292fa4e8 11323456    543525888 System.Collections.Generic.Dictionary`2[[System.Int32, mscorlib],[Microsoft.CodeAnalysis.Diagnostics.AnalysisState+DeclarationAnalyzerStateData, Microsoft.CodeAnalysis]]

https://github.com/dotnet/roslyn/issues/6846 being implemented by @heejaechang for 1.3 should fix this issue - we are moving away from the state tracking model for analyzer execution. However, we still need a rolling stress test for following combinations:

• [x] Solution with large number of projects (150+ in https://github.com/dotnet/roslyn/issues/10365)
• [ ] Solution with a single very large project (~40k files as shared by an internal customer \<%internal_share%>\public\mavasani\OOM_Repro\test.zip)
• [x] 100s to 1000s of analyzers installed – StyleCop, Resharper, FxCop analyzers, etc.

[mavasani]

@basoundr this might overlap with your work for https://github.com/dotnet/roslyn/issues/621, but I am still creating a separate test issue to explicitly track that we add a stress test for customer reported issue in #10365 - we should be able to repro the high memory usage in Update2 bits and confirm that this gets fixed once #6846 has been implemented.

[mavasani]

We should also add a test to cover the issue in https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_workitems?id=214414&fullScreen=false&_a=edit

[mavasani]

I am moving this to the performance team - The only pending test to be added is Solution with a single very large project and this is not specific to analyzers - it more Roslyn wide stress test. We probably need a rolling stress test for typing, intellisense, diagnostics, etc. for this scenario.

[Pilchie]

@mavasani can you re-triage? This was in the 2.0 milestone.