Open SkadOrdos opened 10 months ago
coreclr!CLREventBase::Set [inlined in coreclr!ObjectNative::PulseAll+0x126]: 00007ffa27e587a6 488b09 mov rcx,qword ptr [rcx] ds:000096ac68d2d80a=????????????????
Duplicate of https://github.com/dotnet/runtime/issues/44071 ?
Tagging subscribers to this area: @mangod9 See info in area-owners.md if you want to be subscribed.
Author: | SkadOrdos |
---|---|
Assignees: | - |
Labels: | `area-System.Threading`, `untriaged`, `needs-area-label` |
Milestone: | - |
coreclr!CLREventBase::Set [inlined in coreclr!ObjectNative::PulseAll+0x126]: 00007ffa27e587a6 488b09 mov rcx,qword ptr [rcx] ds:000096ac68d2d80a=????????????????
Duplicate of #44071 ?
Im not sure that is the same crashes, but in our app we had error after migrate to NET. Besides we also sometimes got a crash in ntdll.dll:
FILE_IN_CAB: server.exe.19416.dmp
NTGLOBALFLAG: 0
PROCESS_BAM_CURRENT_THROTTLED: 0
PROCESS_BAM_PREVIOUS_THROTTLED: 0
APPLICATION_VERIFIER_FLAGS: 0
CONTEXT: (.ecxr)
rax=00000000000002e0 rbx=00000310c6722000 rcx=0000000000000008
rdx=00000000000002f0 rsi=0000000000001948 rdi=0000000000000000
rip=00007ffa574bce4b rsp=000000ff0547f4b0 rbp=000000ff0547f529
r8=00007ffa573bbf5a r9=000002d0b2c50000 r10=0000000000000000
r11=00000000000002e0 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=00000310c67220f0
iopl=0 nv up ei pl nz na pe nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
ntdll!TppRaiseHandleStatus+0x8b:
00007ffa574bce4b 488b4d47 mov rcx,qword ptr [rbp+47h] ss:000000ff
0547f570=0000f4069729cfe0
Resetting default scope
EXCEPTION_RECORD: (.exr -1) ExceptionAddress: 00007ffa574bce4b (ntdll!TppRaiseHandleStatus+0x000000000000008b) ExceptionCode: c000070a ExceptionFlags: 00000081 NumberParameters: 5 Parameter[0]: ffffffffc0000008 Parameter[1]: 0000000000001948 Parameter[2]: 00000310c6722000 Parameter[3]: 0000000000000000 Parameter[4]: 00007ffa54ba5590
PROCESS_NAME: server.exe
HANDLE: 0000000000001948 (!handle 0000000000001948)
THREADPOOL_WAITER: !tp wait 310c6722000 Unrecognized command or argument 'wait' Unrecognized command or argument '310c6722000'
threadpool: Displays info about the runtime thread pool.
Usage:
!ext threadpool [options]
Options: -hc, -ti Print the hill climbing log. -wi Print all work items that are queued.
ERROR_CODE: (NTSTATUS) 0xc000070a - 0x
EXCEPTION_CODE_STR: c000070a
EXCEPTION_PARAMETER1: ffffffffc0000008
EXCEPTION_PARAMETER2: 0000000000001948
EXCEPTION_PARAMETER3: 00000310c6722000
EXCEPTION_PARAMETER4: 0
STACK_TEXT:
000000ff0547f4b0 00007ffa
5744d41f : 0000031000000000 00007ffa
54bc0501 0000000000000000 00007ffa
573a6953 : ntdll!TppRaiseHandleStatus+0x8b
000000ff0547f590 00007ffa
573a67e6 : 00000310c67223c0 00000000
00000000 0000000000000000 00000000
00001948 : ntdll!TppSetupNextWait+0xa6a8b
000000ff0547f5e0 00007ffa
573a8729 : 000002d0b12ad8f0 00000000
00000000 00000310c6722188 00000000
00000000 : ntdll!TpSetWaitEx+0xa6
000000ff0547f620 00007ffa
573a47ce : 000002d0b12ad8f0 000002d0
b12ad8f0 0000000000000000 00000310
c6722188 : ntdll!RtlpTpWaitCheckReset+0x41
000000ff0547f650 00007ffa
573a6e52 : 00000310c6722000 000002d0
b12ad8f0 0000000000000000 00007ffa
00000000 : ntdll!RtlpTpWaitCallback+0x2e
000000ff0547f6b0 00007ffa
573ab8e8 : 00000310c6722188 00000310
c66cc030 0000000000000000 00000000
00000000 : ntdll!TppExecuteWaitCallback+0xae
000000ff0547f700 00007ffa
56ee4de0 : 0000000000000000 00000000
00000000 0000000000000000 00000000
00000000 : ntdll!TppWorkerThread+0x448
000000ff0547f9f0 00007ffa
5741ed9b : 0000000000000000 00000000
00000000 0000000000000000 00000000
00000000 : kernel32!BaseThreadInitThunk+0x10
000000ff0547fa20 00000000
00000000 : 0000000000000000 00000000
00000000 0000000000000000 00000000
00000000 : ntdll!RtlUserThreadStart+0x2b
SYMBOL_NAME: ntdll!TppRaiseHandleStatus+8b
MODULE_NAME: ntdll
IMAGE_NAME: ntdll.dll
STACK_COMMAND: ~297s; .ecxr ; kb
FAILURE_BUCKET_ID: THREADPOOL_HANDLE_EXCEPTION_c000070a_ntdll.dll!TppRaiseHandleStatus
Both errors registered as code 0x80131506
are you able to share a dump and/or a repro?
are you able to share a dump and/or a repro?
Unfortunately not, I couldn't share the dump by security policy of company. And we couldn't find fixed way to reproduce the crash - the only thing that clearly defined its depend on NET and no ocurus with Net Framework
Description
Our Application crashed with a internal NetCore Runtime error It began after migrate from Net Framework 4.8 to NET 7. Crash reproduce on NET 7.0.14 / 7.0.15 / 8.0.1
Faulting application name: server.exe, version: 1.0.10.1, time stamp: 0x65410000 Faulting module name: coreclr.dll, version: 8.0.123.58001, time stamp: 0x65684720 Exception code: 0xc0000005 Fault offset: 0x00000000000e87a6 Faulting process id: 0x48f8 Faulting application start time: 0x01da488073edf4b2 Faulting application path: D:\1.1.Server\server.exe Faulting module path: C:\Program Files\dotnet\shared\Microsoft.NETCore.App\8.0.1\coreclr.dll Report Id: be1a8f26-efe5-4a91-9f2f-242672b1661e Faulting package full name: Faulting package-relative application ID:
Description: The process was terminated due to an internal error in the .NET Runtime at IP 0x00007FFA27E587A6 (0x00007FFA27D70000) with exit code 0x80131506.
Reproduction Steps
It crashes spontaneously: could be once a hour or few days without errors. By stack trace it seem depends on Task.Start
Expected behavior
Work without spontaneously crashes
Actual behavior
Dump analyze via WinDbg
***** Preparing the environment for Debugger Extensions Gallery repositories ** ExtensionRepository : Implicit UseExperimentalFeatureForNugetShare : true AllowNugetExeUpdate : true AllowNugetMSCredentialProviderInstall : true AllowParallelInitializationOfLocalRepositories : true
-- Configuring repositories ----> Repository : LocalInstalled, Enabled: true ----> Repository : UserExtensions, Enabled: true
***** Waiting for Debugger Extensions Gallery to Initialize **
Microsoft (R) Windows Debugger Version 10.0.25921.1001 AMD64 Copyright (c) Microsoft Corporation. All rights reserved.
Loading Dump File [H:\server.exe.18680.dmp] User Mini Dump File with Full Memory: Only application data is available
***** Path validation summary ** Response Time (ms) Location Deferred srv Symbol search path is: srv Executable search path is: Windows 10 Version 20348 MP (16 procs) Free x64 Product: Server, suite: TerminalServer SingleUserTS Edition build lab: 20348.1.amd64fre.fe_release.210507-1500 Debug session time: Tue Jan 16 15:58:19.000 2024 (UTC + 2:00) System Uptime: 10 days 8:20:29.450 Process Uptime: 0 days 0:25:50.000 ................................................................ ................................................................ ..................... Loading unloaded module list . This dump file has an exception of interest stored in it. The stored exception information can be accessed via .ecxr. (48f8.4508): Access violation - code c0000005 (first/second chance not available) For analysis of this file, run !analyze -v coreclr!CLREventBase::Set [inlined in coreclr!ObjectNative::PulseAll+0x126]: 00007ffa
27e587a6 488b09 mov rcx,qword ptr [rcx] ds:000096ac
68d2d80a=???????????????? 0:904> !analyze -vFailed to request MethodData, not in JIT code range
KEY_VALUES_STRING: 1
FILE_IN_CAB: server.exe.18680.dmp
NTGLOBALFLAG: 0
PROCESS_BAM_CURRENT_THROTTLED: 0
PROCESS_BAM_PREVIOUS_THROTTLED: 0
APPLICATION_VERIFIER_FLAGS: 0
CONTEXT: (.ecxr) rax=000000ab838fef10 rbx=000000ab829ff200 rcx=000096ac68d2d80a rdx=000000ab838fef30 rsi=000000ab829fea70 rdi=000000ab829fed10 rip=00007ffa27e587a6 rsp=000000ab829ff2d0 rbp=000000ab829ff4a0 r8=0000000000000054 r9=0000000000000056 r10=0000000000000007 r11=0000000000000246 r12=0000000000000000 r13=0000000000000000 r14=0000027c7107f878 r15=0000000000000000 iopl=0 nv up ei pl nz na pe nc cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010200 coreclr!CLREventBase::Set [inlined in coreclr!ObjectNative::PulseAll+0x126]: 00007ffa
27e587a6 488b09 mov rcx,qword ptr [rcx] ds:000096ac
68d2d80a=???????????????? Resetting default scopeEXCEPTION_RECORD: (.exr -1) ExceptionAddress: 00007ffa27e587a6 (coreclr!CLREventBase::Set) ExceptionCode: c0000005 (Access violation) ExceptionFlags: 00000001 NumberParameters: 2 Parameter[0]: 0000000000000000 Parameter[1]: ffffffffffffffff Attempt to read from address ffffffffffffffff
PROCESS_NAME: server.exe
READ_ADDRESS: ffffffffffffffff
ERROR_CODE: (NTSTATUS) 0xc0000005 - 0x%p 0x%p. %s.
EXCEPTION_CODE_STR: c0000005
EXCEPTION_PARAMETER1: 0000000000000000
EXCEPTION_PARAMETER2: ffffffffffffffff
IP_ON_HEAP: 00007ff9c9423f8a The fault address in not in any loaded module, please check your build's rebase log at\bin\build_logs\timebuild\ntrebase.log for module which may
contain the address if it were loaded.
FRAME_ONE_INVALID: 1
STACK_TEXT:
000000ab
829ff2d0 00007ff9
c9423f8a : 0000023c675483a0 0000023c
675483a0 0000023c608a3118 0000023c
67548388 : coreclr!ObjectNative::PulseAll+0x126 000000ab829ff430 00007ff9
ca5b0048 : 0000023c67548388 00007ff9
00000000 000000ab829ff4a0 0000027c
6f03e3d0 : System_Private_CoreLib!System.Threading.ManualResetEventSlim.Set+0x8a 000000ab829ff4b0 00007ff9
ca224447 : 0000023c67548210 00000000
00000000 000000ab829ff540 00000000
00000000 : System_Private_CoreLib!System.Threading.Tasks.Task.RunContinuations+0x128 000000ab829ff550 00007ff9
ca177076 : 0000027c72496170 00000000
00000000 0000027c72496170 00007ffa
56a87563 : System_Private_CoreLib!System.Threading.Tasks.Task.FinishSlow+0x37 000000ab829ff5b0 00007ffa
27eca333 : 0000023c67548210 0000023c
675495d8 0000023c5c822590 00000000
00000000 : System_Private_CoreLib!System.Threading.Tasks.Task.ExecuteWithThreadLocal+0x116 000000ab829ff660 00007ffa
27d84cb4 : 0000000000000008 00000000
00000130 000000ab829ff770 00007ffa
00000000 : coreclr!CallDescrWorkerInternal+0x83 000000ab829ff6a0 00007ffa
27eb74f3 : 0000023c5c81a250 00000000
00000000 0000027c6f03df7c 00000000
00000000 : coreclr!DispatchCallSimple+0x60 000000ab829ff730 00007ffa
27e589dd : 0000027c6f03df70 00007ffa
27e59308 0000027c00004508 0000023c
5c820008 : coreclr!ThreadNative::KickOffThread_Worker+0x63 000000ab829ff790 00007ffa
27e588f3 : 000000ab829ff808 0000023c
5c853fe0 0000000000000000 00000000
00000000 : coreclr!ManagedThreadBase_DispatchMiddle+0x85 000000ab829ff870 00007ffa
27e58a8e : 0000027c00000001 ffffffff
ffffffff 0000023c5c822590 0000023c
5c822590 : coreclr!ManagedThreadBase_DispatchOuter+0xab 000000ab829ff910 00007ffa
56ee4de0 : 0000000000000000 00000000
00000000 0000000000000000 00000000
00000000 : coreclr!ThreadNative::KickOffThread+0x7e 000000ab829ff970 00007ffa
5741ed9b : 0000000000000000 00000000
00000000 0000000000000000 00000000
00000000 : kernel32!BaseThreadInitThunk+0x10 000000ab829ff9a0 00000000
00000000 : 0000000000000000 00000000
00000000 0000000000000000 00000000
00000000 : ntdll!RtlUserThreadStart+0x2bFAULTING_SOURCE_LINE: D:\a_work\1\s\src\coreclr\classlibnative\bcltype\objectnative.cpp
FAULTING_SOURCE_FILE: D:\a_work\1\s\src\coreclr\classlibnative\bcltype\objectnative.cpp
FAULTING_SOURCE_LINE_NUMBER: 340
FAULTING_SOURCE_SRV_COMMAND: https://raw.githubusercontent.com/dotnet/runtime/bf5e279d9239bfef5bb1b8d6212f1b971c434606/src/coreclr/classlibnative/bcltype/objectnative.cpp
FAULTING_SOURCE_CODE:
No source found for 'D:\a_work\1\s\src\coreclr\vm\synch.cpp'
SYMBOL_NAME: coreclr!ObjectNative::PulseAll+126
MODULE_NAME: coreclr
IMAGE_NAME: coreclr.dll
STACK_COMMAND: ~904s; .ecxr ; kb
FAILURE_BUCKET_ID: INVALID_POINTER_READ_c0000005_coreclr.dll!ObjectNative::PulseAll
OS_VERSION: 10.0.20348.1
BUILDLAB_STR: fe_release
OSPLATFORM_TYPE: x64
OSNAME: Windows 10
IMAGE_VERSION: 8.0.123.58001
FAILURE_ID_HASH: {be7e1d5f-f864-f771-a72a-7be333bc8c49}
Followup: MachineOwner
Regression?
We dont have crash on Framework 4.5 / 4.8, but it reproduced on NET 7 / 8
Known Workarounds
Service MySQL56
Configuration
NET 8.0.1, VM with Windows Server 2022 x64, 48 GB RAM
Other information
No response