Native SGen crash after updating from 8004 to 8053

After updating ios workload from 17.2.8004 to 17.2.8053 we started to get native crashes with concurrent SGen. Our application is relatively big game and we could not reproduce it on some empty project. However with the same appdata we get a 100% repro at almost the same spot of reading a big json object with our custom json reader. We tried to update dotnet from 8.0.100 to 8.0.104 but unfortunately it doesn't help, so it's definately a workload issue somwhere between 8004 and 8053. Downgrading to 8004 helps but at the same time we need a fix from https://github.com/xamarin/xamarin-macios/issues/19417 that only exist at 8053. Also we have a win and android projects (same game) with the same codebase running latest .net8 and no such issues with them.

Steps to Reproduce

Expected Behavior

No crash

Actual Behavior

Native crash examples crash1.txt crash2.txt

Environment

XCode 15.3. Release mode, AOT. Props for release configuration

<PropertyGroup>
    <OutputType>Exe</OutputType>
    <RuntimeIdentifier>ios-arm64</RuntimeIdentifier>
    <SupportedOSPlatformVersion>11.0</SupportedOSPlatformVersion>
    <TargetFramework>net8.0-ios</TargetFramework>
    <MtouchEnableSGenConc>true</MtouchEnableSGenConc>
    <MtouchFloat32>true</MtouchFloat32>
    <BuildIpa>true</BuildIpa>
    <MtouchLink>SdkOnly</MtouchLink>
    <MtouchUseLlvm>true</MtouchUseLlvm>
    <_ExportSymbolsExplicitly>false</_ExportSymbolsExplicitly>
</PropertyGroup>

Version information

``` dotnet --info .NET SDK: Version: 8.0.104 Commit: 034f91fcc0 Workload version: 8.0.100-manifests.1f2e3bea Runtime Environment: OS Name: Mac OS X OS Version: 14.4 OS Platform: Darwin RID: osx-x64 Base Path: /usr/local/share/dotnet/sdk/8.0.104/ .NET workloads installed: Workload version: 8.0.100-manifests.1f2e3bea [macos] Installation Source: SDK 8.0.100 Manifest Version: 14.2.8053/8.0.100 Manifest Path: /usr/local/share/dotnet/sdk-manifests/8.0.100/microsoft.net.sdk.macos/14.2.8053/WorkloadManifest.json Install Type: FileBased [ios] Installation Source: SDK 8.0.100 Manifest Version: 17.2.8053/8.0.100 Manifest Path: /usr/local/share/dotnet/sdk-manifests/8.0.100/microsoft.net.sdk.ios/17.2.8053/WorkloadManifest.json Install Type: FileBased Host: Version: 8.0.4 Architecture: x64 Commit: 2d7eea2529 .NET SDKs installed: 6.0.418 [/usr/local/share/dotnet/sdk] 6.0.421 [/usr/local/share/dotnet/sdk] 7.0.312 [/usr/local/share/dotnet/sdk] 7.0.315 [/usr/local/share/dotnet/sdk] 8.0.101 [/usr/local/share/dotnet/sdk] 8.0.104 [/usr/local/share/dotnet/sdk] .NET runtimes installed: Microsoft.AspNetCore.App 6.0.26 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.29 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 7.0.15 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 7.0.18 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 8.0.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 8.0.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 6.0.26 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.29 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 7.0.15 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 7.0.18 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 8.0.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 8.0.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Other architectures found: None Environment variables: Not set global.json file: Not found Learn more: https://aka.ms/dotnet/info Download .NET: https://aka.ms/dotnet/download ```

Build Logs

Example Project (If Possible)

Also tried to disable concurrent gc, still same crash but in the main thread now. Here a few:

thread #1, name = 'tid_103', queue = 'com.apple.main-thread', stop reason = signal SIGSEGV
  * frame #0: 0x0000000107d0f080 AppName`drain_gray_stack at sgen-marksweep-drain-gray-stack.h:158:10 [opt]
    frame #1: 0x0000000107d0f07c AppName`drain_gray_stack at sgen-scan-object.h:66:3 [opt]
    frame #2: 0x0000000107d0ec04 AppName`drain_gray_stack at sgen-marksweep-drain-gray-stack.h:347:3 [opt]
    frame #3: 0x0000000107d0eb90 AppName`drain_gray_stack(queue=0x000000016d8582f0) at sgen-marksweep.c:1287:10 [opt]
    frame #4: 0x0000000107d021f8 AppName`finish_gray_stack [inlined] sgen_drain_gray_stack(ctx=ScanCopyContext @ 0x0000600003e77c40) at sgen-gc.c:578:9 [opt]
    frame #5: 0x0000000107d021e8 AppName`finish_gray_stack(generation=1, ctx=ScanCopyContext @ 0x0000600003e77c60) at sgen-gc.c:1140:2 [opt]
    frame #6: 0x0000000107d02d20 AppName`major_finish_collection(gc_thread_gray_queue=0x000000016d8582f0, reason="Minor allowance", is_overflow=0, old_next_pin_slot=280, forced=0) at sgen-gc.c:2323:2 [opt]
    frame #7: 0x0000000107d01aa0 AppName`major_do_collection(reason="Minor allowance", is_overflow=0, forced=0) at sgen-gc.c:2465:2 [opt]
    frame #8: 0x0000000107cfd7cc AppName`sgen_perform_collection [inlined] sgen_perform_collection_inner(requested_size=<unavailable>, generation_to_collect=<unavailable>, reason=<unavailable>, forced_serial=<unavailable>, stw=<unavailable>) at sgen-gc.c:2665:14 [opt]
    frame #9: 0x0000000107cfd500 AppName`sgen_perform_collection(requested_size=4096, generation_to_collect=1, reason="Minor allowance", forced_serial=0, stw=1) at sgen-gc.c:2762:2 [opt]
    frame #10: 0x0000000107cfd49c AppName`sgen_ensure_free_space(size=4096, generation=<unavailable>) at sgen-gc.c:2616:2 [opt]
    frame #11: 0x0000000107cf38f0 AppName`sgen_alloc_obj_nolock(vtable=0x000000010bbcead8, size=992) at sgen-alloc.c:279:6 [opt]
    frame #12: 0x0000000107ddb888 AppName`mono_gc_alloc_vector(vtable=0x000000010bbcead8, size=992, max_length=120) at sgen-mono.c:1119:20 [opt]
    frame #13: 0x0000000107da2ca0 AppName`mono_array_new_full_checked(array_class=0x000000010bbc98f8, lengths=0x000000016d8585f0, lower_bounds=0x000000016d8585f8, error=0x000000016d858608) at object.c:5943:21 [opt]
    frame #14: 0x0000000107d5aa48 AppName`ves_icall_System_Array_InternalCreate(result=0x000000016d858758, type=0x000000010b8d4de0, rank=1, pLengths=<unavailable>, pLowerBounds=<unavailable>) at icall.c:777:12 [opt]
    frame #15: 0x0000000103f156d8 AppName`wrapper_managed_to_native_System_Array_InternalCreate_System_Array__intptr_int_int__int_ + 152
    frame #16: 0x0000000107724ea4 AppName`corlib_System_Array_InternalCreate_System_RuntimeType_int_int__int_ + 88
    frame #17: 0x0000000107725e6c AppName`corlib_System_Array_CreateInstance_System_Type_int + 132
    frame #18: 0x000000010799b23c AppName`corlib_System_Collections_ArrayList_ToArray_System_Type + 64
    frame #19: 0x0000000104df55a4 AppName`AppName_JsonReader_ReadArray_System_Type at JsonReader.cs:606:21 [opt]
    frame #20: 0x0000000104df4ac4 AppName`AppName_JsonReader_Read_System_Type_bool at JsonReader.cs:313:29 [opt]

 thread #1, name = 'tid_103', queue = 'com.apple.main-thread', stop reason = signal SIGSEGV
  * frame #0: 0x0000000105e1f080 AppName`drain_gray_stack at sgen-marksweep-drain-gray-stack.h:158:10 [opt]
    frame #1: 0x0000000105e1f07c AppName`drain_gray_stack at sgen-scan-object.h:66:3 [opt]
    frame #2: 0x0000000105e1ec04 AppName`drain_gray_stack at sgen-marksweep-drain-gray-stack.h:347:3 [opt]
    frame #3: 0x0000000105e1eb90 AppName`drain_gray_stack(queue=0x000000016f747fb0) at sgen-marksweep.c:1287:10 [opt]
    frame #4: 0x0000000105e121f8 AppName`finish_gray_stack [inlined] sgen_drain_gray_stack(ctx=ScanCopyContext @ 0x00006000012ec680) at sgen-gc.c:578:9 [opt]
    frame #5: 0x0000000105e121e8 AppName`finish_gray_stack(generation=1, ctx=ScanCopyContext @ 0x00006000012ec6a0) at sgen-gc.c:1140:2 [opt]
    frame #6: 0x0000000105e12d20 AppName`major_finish_collection(gc_thread_gray_queue=0x000000016f747fb0, reason="Minor allowance", is_overflow=0, old_next_pin_slot=315, forced=0) at sgen-gc.c:2323:2 [opt]
    frame #7: 0x0000000105e11aa0 AppName`major_do_collection(reason="Minor allowance", is_overflow=0, forced=0) at sgen-gc.c:2465:2 [opt]
    frame #8: 0x0000000105e0d7cc AppName`sgen_perform_collection [inlined] sgen_perform_collection_inner(requested_size=<unavailable>, generation_to_collect=<unavailable>, reason=<unavailable>, forced_serial=<unavailable>, stw=<unavailable>) at sgen-gc.c:2665:14 [opt]
    frame #9: 0x0000000105e0d500 AppName`sgen_perform_collection(requested_size=4096, generation_to_collect=1, reason="Minor allowance", forced_serial=0, stw=1) at sgen-gc.c:2762:2 [opt]
    frame #10: 0x0000000105e0d49c AppName`sgen_ensure_free_space(size=4096, generation=<unavailable>) at sgen-gc.c:2616:2 [opt]
    frame #11: 0x0000000105e038f0 AppName`sgen_alloc_obj_nolock(vtable=0x0000000109892fc8, size=24) at sgen-alloc.c:279:6 [opt]
    frame #12: 0x0000000105eebaf4 AppName`mono_gc_alloc_string(vtable=0x0000000109892fc8, size=24, len=1) at sgen-mono.c:1208:21 [opt]
    frame #13: 0x0000000105eabf78 AppName`mono_string_new_size_checked(len=1, error=0x000000016f7482a0) at object.c:6349:6 [opt]
    frame #14: 0x0000000105eb3420 AppName`mono_string_new_size_handle(len=<unavailable>, error=<unavailable>) at object.c:6323:9 [opt]
    frame #15: 0x0000000105e7efe8 AppName`ves_icall_System_String_FastAllocateString_raw(a0=1) at icall-def.h:552:1 [opt]
    frame #16: 0x000000010202dca8 AppName`wrapper_managed_to_native_string_FastAllocateString_int + 120
    frame #17: 0x00000001058663fc AppName`corlib_string_InternalSubString_int_int + 64
    frame #18: 0x0000000105866264 AppName`corlib_string_Substring_int_int + 104
    frame #19: 0x0000000102f06624 AppName`AppName_JsonReader_ReadNumber_System_Type at JsonReader.cs:884:17 [opt]
    frame #20: 0x0000000102f04b20 AppName`AppName_JsonReader_Read_System_Type_bool at JsonReader.cs:321:29 [opt]
    frame #21: 0x0000000102f04f10 AppName`AppName_JsonReader_ReadObject_System_Type at JsonReader.cs:465:21 [opt]

xamarin / xamarin-macios