StackOverflowException from collection initializer with huge number of elements

[JakubLinhart]

Version Used:

Microsoft (R) Visual C# Compiler version 3.7.0-6.20459.4 (7ee7c540) Microsoft Windows 10 Pro 10.0.19041 Build 19041 Runtime version: 4.0.30319.42000

Steps to Reproduce:

Try to initialize a list with huge number of elements (~10k) each allocating a value object. For example:

var cityCodes = new List<SomethingSomethingInfo>() {
    new SomethingSomethingInfo("Cleoside", "ji41 7qi", "Jamaica", new DateTime(1900, 1, 1), new DateTime(2000, 1, 1)),
    new SomethingSomethingInfo("North Margaretstad", "ck3 8of", "Libyan Arab Jamahiriya", new DateTime(1900, 1, 1), new DateTime(2000, 1, 1)),
    new SomethingSomethingInfo("Estefaniaside", "ei3 5ye", "Saint Lucia", new DateTime(1900, 1, 1), new DateTime(2000, 1, 1)),
    :
    :

A complete example project is in this repo. It seems like a crazy thing to create such huge initializers but it is much faster than deserializing any imaginable data format.

Expected Behavior:

Creates a List instance with all elements successfully.

Actual Behavior:

Throws StackOverflowException.

It seems like JIT allocates space for all instantiated value types:

00007FFDF1130080  push        rbp  
00007FFDF1130081  push        rdi  
00007FFDF1130082  push        rsi  
00007FFDF1130083  xor         eax,eax  
00007FFDF1130085  test        qword ptr [rsp+rax],rax  
00007FFDF1130089  sub         rax,1000h  
00007FFDF113008F  cmp         rax,0FFFFFFFFFFEDB030h  // = -1200080
00007FFDF1130095  jge         Something.SomethingSomethingCodes.Initialize()+05h (07FFDF1130085h)  

If this space is larger than available stack size, then runtime naturally throws StackOverflowException even before it starts executing a containing method. I wonder what is the reason for the up front stack space allocation in such case.

It seems like this issue is related more to JIT than to Roslyn, but I would like to know if there is some workaround in C# that would allow initializers with huge number of elements without increasing the default stack size. Also even if it is a runtime limitation, would it be possible to bridge the problem in C# compiler?

[CyrusNajmabadi]

but I would like to know if there is some workaround in C# that would allow initializers with huge number of elements

Sounds like you could easily just break this into a few helper methods that kept the amount reasonable. i.e. only adding a couple thousand items at a time.

[CyrusNajmabadi]

Also, looks like you have a loooot of temps there. For values that are repeated. Can you just extract things like new DateTime(1900, 1, 1) to temps that you then reference as appropriate?

[JakubLinhart]

Good point. Unfortunately the real use case has no repeated values. Example project fixed.

[RikkiGibson]

I would consider doing something like generating a local function for each 1000 elements you want to add to the list.

[GrabYourPitchforks]

Sample project targets 4.7.2. However, also repros on net5.0.

[AndyAyersMS]

Similar issue to #8980.

[BruceForstall]

Given that there is a related item for the JIT (as well as other items related to doing "stack packing" which might help in this case), I'm going to close this as a codegen issue.