Open stephentoub opened 1 year ago
Crashing on linux-arm64. Here's a managed stack trace:
at <unknown> <0xffffffff>
at System.Object:__icall_wrapper_mono_marshal_isinst_with_cache <0x00007>
at System.Object:__isinst_with_cache <0x00083>
at System.Collections.Immutable.ImmutableSortedDictionary`2:TryCastToImmutableMap <0x000cf>
at System.Collections.Immutable.ImmutableSortedDictionary`2:FillFromEmpty <0x000a3>
at System.Collections.Immutable.ImmutableSortedDictionary`2:AddRange <0x000bb>
at System.Collections.Immutable.ImmutableSortedDictionary`2:AddRange <0x00063>
at System.Collections.Immutable.ImmutableSortedDictionary:ToImmutableSortedDictionary <0x0028f>
at System.Collections.Immutable.ImmutableSortedDictionary:ToImmutableSortedDictionary <0x0006f>
at Microsoft.CodeAnalysis.TextDocumentStates`1:.ctor <0x002f7>
at Microsoft.CodeAnalysis.ProjectState:.ctor <0x006fb>
at Microsoft.CodeAnalysis.SolutionState:AddProject <0x000f3>
at Microsoft.CodeAnalysis.Solution:AddProject <0x0003f>
at Microsoft.CodeAnalysis.Solution:AddProject <0x000b3>
at Microsoft.CodeAnalysis.Solution:AddProject <0x00057>
at SourceGenerators.Tests.RoslynTestUtils:CreateTestProject <0x003f7>
at <RunGenerator>d__8:MoveNext <0x000fb>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x0008b>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start <0x0003b>
at SourceGenerators.Tests.RoslynTestUtils:RunGenerator <0x0021b>
at <TestEmitter>d__0:MoveNext <0x004f3>
at AsyncStateMachineBox`1:ExecutionContextCallback <0x00043>
at System.Threading.ExecutionContext:RunInternal <0x000bf>
at AsyncStateMachineBox`1:MoveNext <0x000f3>
at AsyncStateMachineBox`1:MoveNext <0x0002f>
at System.Threading.Tasks.AwaitTaskContinuation:RunOrScheduleAction <0x00107>
at System.Threading.Tasks.Task:RunContinuations <0x0022f>
at System.Threading.Tasks.Task:FinishContinuations <0x0009b>
at System.Threading.Tasks.Task`1:TrySetResult <0x00103>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetExistingTaskResult <0x000cf>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetResult <0x000b3>
at <InternalReadAllTextAsync>d__86:MoveNext <0x008cf>
at AsyncStateMachineBox`1:ExecutionContextCallback <0x0003b>
at System.Threading.ExecutionContext:RunInternal <0x000bf>
at AsyncStateMachineBox`1:MoveNext <0x000eb>
at AsyncStateMachineBox`1:MoveNext <0x0002f>
at System.Threading.Tasks.AwaitTaskContinuation:RunOrScheduleAction <0x00107>
at System.Threading.Tasks.Task:RunContinuations <0x0022f>
at System.Threading.Tasks.Task:FinishContinuations <0x0009b>
at System.Threading.Tasks.Task`1:TrySetResult <0x000bf>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetExistingTaskResult <0x0009b>
at System.Runtime.CompilerServices.AsyncValueTaskMethodBuilder`1:SetResult <0x0008f>
at <ReadAsyncInternal>d__69:MoveNext <0x01ecb>
at AsyncStateMachineBox`1:ExecutionContextCallback <0x0003b>
at System.Threading.ExecutionContext:RunInternal <0x000bf>
at AsyncStateMachineBox`1:MoveNext <0x000eb>
at AsyncStateMachineBox`1:MoveNext <0x0002f>
at System.Threading.Tasks.AwaitTaskContinuation:RunOrScheduleAction <0x00107>
at System.Threading.Tasks.Task:RunContinuations <0x0022f>
at System.Threading.Tasks.Task:FinishContinuations <0x0009b>
at System.Threading.Tasks.Task`1:TrySetResult <0x000bf>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetExistingTaskResult <0x0009b>
at System.Runtime.CompilerServices.AsyncValueTaskMethodBuilder`1:SetResult <0x0008f>
at <ReadBufferAsync>d__72:MoveNext <0x00cbb>
at AsyncStateMachineBox`1:ExecutionContextCallback <0x0003b>
at System.Threading.ExecutionContext:RunInternal <0x000bf>
at AsyncStateMachineBox`1:MoveNext <0x000eb>
at AsyncStateMachineBox`1:MoveNext <0x0002f>
at <>c:<.cctor>b__49_0 <0x00093>
at System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1:SignalCompletion <0x0011f>
at System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1:SetResult <0x0003b>
at StateMachineBox:SetResult <0x00043>
at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1:SetResult <0x00097>
at <ReadAsyncSlowPath>d__37:MoveNext <0x01cdb>
at StateMachineBox`1:ExecutionContextCallback <0x0003b>
at System.Threading.ExecutionContext:RunInternal <0x000bf>
at StateMachineBox`1:MoveNext <0x00067>
at <>c:<.cctor>b__49_0 <0x00093>
at System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1:SignalCompletion <0x0011f>
at System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1:SetResult <0x0003b>
at ThreadPoolValueTaskSource:ExecuteInternal <0x006d7>
at ThreadPoolValueTaskSource:System.Threading.IThreadPoolWorkItem.Execute <0x00057>
at System.Threading.ThreadPoolWorkQueue:Dispatch <0x003ef>
at WorkerThread:WorkerThreadStart <0x00193>
at System.Threading.Thread:StartCallback <0x000e7>
at System.Object:runtime_invoke_void__this__ <0x00087>
Here's the corresponding native thread:
#0 0x0000ff5f546c8d5c in __waitpid (pid=<optimized out>, stat_loc=0xff5f48a244a0, options=<optimized out>) at ../sysdeps/unix/sysv/linux/waitpid.c:30
#1 0x0000ff5f53f4dc80 in mono_dump_native_crash_info () from /root/helix/work/correlation/shared/Microsoft.NETCore.App/8.0.0/libcoreclr.so
#2 0x0000ff5f53effa1c in mono_handle_native_crash () from /root/helix/work/correlation/shared/Microsoft.NETCore.App/8.0.0/libcoreclr.so
#3 0x0000ff5f53e6e8c8 in mono_sigsegv_signal_handler_debug () from /root/helix/work/correlation/shared/Microsoft.NETCore.App/8.0.0/libcoreclr.so
#4 <signal handler called>
#5 0x0000ff5f5407ff54 in mono_object_handle_isinst () from /root/helix/work/correlation/shared/Microsoft.NETCore.App/8.0.0/libcoreclr.so
#6 0x0000ff5f54080088 in mono_object_isinst_checked () from /root/helix/work/correlation/shared/Microsoft.NETCore.App/8.0.0/libcoreclr.so
#7 0x0000ff5f540514a8 in mono_marshal_isinst_with_cache () from /root/helix/work/correlation/shared/Microsoft.NETCore.App/8.0.0/libcoreclr.so
#8 0x0000ff5f53955e8c in ?? ()
#9 0x0000ff5f340031d0 in ?? ()
The other crash has a slightly different managed stack trace:
at <unknown> <0xffffffff>
at System.Object:__icall_wrapper_mono_marshal_isinst_with_cache <0x00007>
at System.Object:__isinst_with_cache <0x00083>
at System.Collections.Immutable.ImmutableExtensions:TryGetCount <0x0015b>
at System.Collections.Immutable.ImmutableExtensions:TryGetCount <0x0005b>
at Builder:AddRange <0x00077>
at Microsoft.CodeAnalysis.PooledObjects.ArrayBuilder`1:AddRange <0x0003f>
at Microsoft.CodeAnalysis.CSharp.Symbol:GetAttributesToBind <0x002db>
at Microsoft.CodeAnalysis.CSharp.Symbol:LoadAndValidateAttributes <0x000c7>
at Microsoft.CodeAnalysis.CSharp.Symbols.SourceMethodSymbolWithAttributes:GetAttributesBag <0x001d7>
at Microsoft.CodeAnalysis.CSharp.Symbols.SourceMethodSymbolWithAttributes:GetAttributesBag <0x0006b>
at Microsoft.CodeAnalysis.CSharp.Symbols.SourceMethodSymbolWithAttributes:GetAttributes <0x0002b>
at Microsoft.CodeAnalysis.CSharp.Symbols.PublicModel.Symbol:Microsoft.CodeAnalysis.ISymbol.GetAttributes <0x00053>
at Microsoft.CodeAnalysis.SyntaxValueProvider:<ForAttributeWithMetadataName>g__getMatchingAttributes|11_1 <0x001a3>
at <>c__DisplayClass11_0`1:<ForAttributeWithMetadataName>b__0 <0x0031f>
at <>c__DisplayClass0_0`2:<WrapUserFunction>b__0 <0x0007f>
at Microsoft.CodeAnalysis.TransformNode`2:UpdateStateTable <0x0039b>
at Builder:GetLatestStateTableForNode <0x000df>
at Microsoft.CodeAnalysis.TransformNode`2:UpdateStateTable <0x0008b>
at Builder:GetLatestStateTableForNode <0x000df>
at Microsoft.CodeAnalysis.TransformNode`2:UpdateStateTable <0x0008b>
at Builder:GetLatestStateTableForNode <0x000df>
at Microsoft.CodeAnalysis.TransformNode`2:UpdateStateTable <0x0008b>
at Builder:GetLatestStateTableForNode <0x000df>
at Microsoft.CodeAnalysis.TransformNode`2:UpdateStateTable <0x0008b>
at Builder:GetLatestStateTableForNode <0x000df>
at Microsoft.CodeAnalysis.TransformNode`2:UpdateStateTable <0x0008b>
at Builder:GetLatestStateTableForNode <0x000df>
at Microsoft.CodeAnalysis.SourceOutputNode`1:UpdateStateTable <0x0010b>
at Builder:GetLatestStateTableForNode <0x000d7>
at Microsoft.CodeAnalysis.SourceOutputNode`1:AppendOutputs <0x0008b>
at Microsoft.CodeAnalysis.GeneratorDriver:UpdateOutputs <0x00283>
at Microsoft.CodeAnalysis.GeneratorDriver:RunGeneratorsCore <0x017ef>
at Microsoft.CodeAnalysis.GeneratorDriver:RunGenerators <0x00047>
at <AddingFileWithNewLibraryImport_DoesNotRegenerateOriginalMethod>d__3:MoveNext <0x003d7>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x0008b>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder:Start <0x00057>
at LibraryImportGenerator.UnitTests.IncrementalGenerationTests:AddingFileWithNewLibraryImport_DoesNotRegenerateOriginalMethod <0x0012b>
at <Module>:runtime_invoke_object__this__ <0x00087>
at <unknown> <0xffffffff>
at System.Reflection.RuntimeMethodInfo:InternalInvoke <0x00007>
at System.Reflection.MethodBaseInvoker:InterpretedInvoke_Method <0x00073>
at System.Reflection.MethodBaseInvoker:InvokeWithNoArgs <0x00093>
at System.Reflection.RuntimeMethodInfo:Invoke <0x001df>
at System.Reflection.MethodBase:Invoke <0x00053>
at Xunit.Sdk.TestInvoker`1:CallTestMethod <0x00047>
at <<InvokeTestMethodAsync>b__1>d:MoveNext <0x0037f>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x0009f>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder:Start <0x0002f>
at <>c__DisplayClass48_0:<InvokeTestMethodAsync>b__1 <0x000cb>
at <AggregateAsync>d__4:MoveNext <0x000c7>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x0009b>
at Xunit.Sdk.ExecutionTimer:AggregateAsync <0x000eb>
at <>c__DisplayClass48_0:<InvokeTestMethodAsync>b__0 <0x0015b>
at <RunAsync>d__9:MoveNext <0x00067>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x00093>
at Xunit.Sdk.ExceptionAggregator:RunAsync <0x000e7>
at <InvokeTestMethodAsync>d__48:MoveNext <0x0028b>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x000a7>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start <0x0002f>
at Xunit.Sdk.TestInvoker`1:InvokeTestMethodAsync <0x00113>
at Xunit.Sdk.XunitTestInvoker:InvokeTestMethodAsync <0x0011b>
at <<RunAsync>b__47_0>d:MoveNext <0x0073b>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x000af>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start <0x0002f>
at Xunit.Sdk.TestInvoker`1:<RunAsync>b__47_0 <0x000d3>
at <RunAsync>d__10`1:MoveNext <0x00073>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x00093>
at Xunit.Sdk.ExceptionAggregator:RunAsync <0x000e7>
at Xunit.Sdk.TestInvoker`1:RunAsync <0x000e7>
at Xunit.Sdk.XunitTestRunner:InvokeTestMethodAsync <0x000cb>
at <InvokeTestAsync>d__4:MoveNext <0x001c3>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x000a3>
at Xunit.Sdk.XunitTestRunner:InvokeTestAsync <0x000ef>
at <>c__DisplayClass43_0:<RunAsync>b__0 <0x00043>
at <RunAsync>d__10`1:MoveNext <0x0008b>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x0009f>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start <0x0002f>
at Xunit.Sdk.ExceptionAggregator:RunAsync <0x0011b>
at <RunAsync>d__43:MoveNext <0x0048b>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x000af>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start <0x0002f>
at Xunit.Sdk.TestRunner`1:RunAsync <0x000d3>
at Xunit.Sdk.XunitTestCaseRunner:RunTestAsync <0x000df>
at <RunAsync>d__19:MoveNext <0x00383>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x000a7>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start <0x0002f>
at Xunit.Sdk.TestCaseRunner`1:RunAsync <0x000cf>
at Xunit.Sdk.XunitTestCase:RunAsync <0x000b3>
at Xunit.Sdk.XunitTestMethodRunner:RunTestCaseAsync <0x0009f>
at <RunTestCasesAsync>d__32:MoveNext <0x001cb>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x000af>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start <0x0002f>
at Xunit.Sdk.TestMethodRunner`1:RunTestCasesAsync <0x000d3>
at <RunAsync>d__31:MoveNext <0x001a7>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x0009f>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start <0x0002f>
at Xunit.Sdk.TestMethodRunner`1:RunAsync <0x000cb>
at Xunit.Sdk.XunitTestClassRunner:RunTestMethodAsync <0x000df>
at <RunTestMethodsAsync>d__38:MoveNext <0x007ab>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x000b7>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start <0x0002f>
at Xunit.Sdk.TestClassRunner`1:RunTestMethodsAsync <0x000d7>
at <RunAsync>d__37:MoveNext <0x003a3>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x000a7>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start <0x0002f>
at Xunit.Sdk.TestClassRunner`1:RunAsync <0x000cf>
at Xunit.Sdk.XunitTestCollectionRunner:RunTestClassAsync <0x000e3>
at <RunTestClassesAsync>d__28:MoveNext <0x003cb>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x000af>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start <0x0002f>
at Xunit.Sdk.TestCollectionRunner`1:RunTestClassesAsync <0x000d3>
at <RunAsync>d__27:MoveNext <0x003a3>
at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x000a7>
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start <0x0002f>
at Xunit.Sdk.TestCollectionRunner`1:RunAsync <0x000cf>
at Xunit.Sdk.XunitTestAssemblyRunner:RunTestCollectionAsync <0x000af>
at <>c__DisplayClass14_2:<RunTestCollectionsAsync>b__2 <0x0006f>
at System.Threading.Tasks.Task`1:InnerInvoke <0x0006f>
at <>c:<.cctor>b__281_0 <0x0003b>
at System.Threading.ExecutionContext:RunInternal <0x000bf>
at System.Threading.Tasks.Task:ExecuteWithThreadLocal <0x00257>
at System.Threading.Tasks.Task:ExecuteEntry <0x000bb>
at <>c:<.cctor>b__8_0 <0x00067>
at Xunit.Sdk.MaxConcurrencySyncContext:RunOnSyncContext <0x0004f>
at <>c__DisplayClass11_0:<WorkerThreadProc>b__0 <0x00053>
at System.Threading.ExecutionContext:RunInternal <0x000bf>
at System.Threading.ExecutionContext:Run <0x00047>
at System.Object:lambda_method2 <0x0008b>
at Xunit.Sdk.ExecutionContextHelper:Run <0x00063>
at Xunit.Sdk.MaxConcurrencySyncContext:WorkerThreadProc <0x00223>
at <>c:<QueueUserWorkItem>b__5_0 <0x00077>
at System.Threading.Tasks.Task:InnerInvoke <0x000b7>
at <>c:<.cctor>b__281_0 <0x0003b>
at System.Threading.ExecutionContext:RunInternal <0x000bf>
at System.Threading.Tasks.Task:ExecuteWithThreadLocal <0x00257>
at System.Threading.Tasks.Task:ExecuteEntryUnsafe <0x000af>
at <>c:<.cctor>b__10_0 <0x0006b>
at System.Threading.Thread:StartCallback <0x0012b>
at System.Object:runtime_invoke_void__this__ <0x00087>
@vargaz both internal static bool TryGetCount<T>(this IEnumerable<T> sequence, out int count)
and private static bool TryCastToImmutableMap(IEnumerable<KeyValuePair<TKey, TValue>> sequence, [NotNullWhen(true)] out ImmutableSortedDictionary<TKey, TValue>? other)
are doing casts of some IEnumerable<T>
objects. (and I guess this would be in gshared) did we change anything there recently?
Don't think so, this is probably just a random crash.
Doesn't seem like a deterministic failure, moving to 9.0.0, but will evaluate a backport of fix when ready.
I think this is the same as https://github.com/dotnet/runtime/issues/87444 which I opened back in June, isn't it? Which, by the way, I reported to be happening since 7.0.
Another Log
Similar stack trace
Also in https://github.com/dotnet/runtime/pull/99332 in System.CodeDom.Tests on arm64 linux
@vargaz are we maybe missing a barrier for the isinst cache variable?
Since this is still getting hit, I'm going to do the bump dance and we should try to address in 10 (or sooner).
Build Information
Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=363089 Build error leg or test failing: Microsoft.Extensions.Logging.Generators.Roslyn4.0.Tests.WorkItemExecution Pull request: https://github.com/dotnet/runtime/pull/89992
Error Message
Fill the error message using step by step known issues guidance.
Known issue validation
Build: :mag_right: https://dev.azure.com/dnceng-public/public/_build/results?buildId=363089 Error message validated:
System.Object:__icall_wrapper_mono_marshal_isinst_with_cache
Result validation: :white_check_mark: Known issue matched with the provided build. Validation performed at: 8/4/2023 1:48:24 PM UTCReport
Summary