search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.17k stars 4.72k forks source link

High CPU utilization on coreclr_shutdown_2 #13275

Closed duskembayev closed 4 years ago

duskembayev commented 5 years ago

Hi all. I have an application with some security limits that executes managed .NET Core library (using custom CLR hosting). Everything works fine, but I found that process utilization permanently about 20%. After investigation, using ProcessExplorer, I found thread with this stack trace:

ntoskrnl.exe!KeSynchronizeExecution+0x5aa6
ntoskrnl.exe!KeWaitForMutexObject+0x1422
ntoskrnl.exe!KeWaitForMutexObject+0xab4
ntoskrnl.exe!KeWaitForMutexObject+0x255
ntoskrnl.exe!KeGetProcessorNumberFromIndex+0xfc7
ntoskrnl.exe!KeWaitForMutexObject+0x34b8
ntoskrnl.exe!KiCheckForKernelApcDelivery+0x2b
ntoskrnl.exe!KeLeaveCriticalRegion+0x37
ntoskrnl.exe!SeUnlockSubjectContext+0x1b
ntoskrnl.exe!ObOpenObjectByPointerWithTag+0x6e8
ntoskrnl.exe!SeUnlockSubjectContext+0x1563
ntoskrnl.exe!ObOpenObjectByNameEx+0x201
ntoskrnl.exe!NtCreateFile+0x48a
ntoskrnl.exe!IoCreateFile+0x8a
ntoskrnl.exe!MmResetDriverPaging+0x44b
ntoskrnl.exe!setjmpex+0x7b55
ntdll.dll!ZwCreateNamedPipeFile+0x14
KERNELBASE.dll!CreateNamedPipeW+0x1bb
KERNEL32.DLL!CreateNamedPipeA+0x6f
coreclr.dll!coreclr_shutdown_2+0x5666
coreclr.dll!coreclr_shutdown_2+0x5606
KERNEL32.DLL!BaseThreadInitThunk+0x14
ntdll.dll!RtlUserThreadStart+0x21

Looks like coreclr cannot create named pipe (named pipes are limited by restricted token of the process). Could you help me with this problem?

dotnet version: 3.0.100-preview8-013656 os: win-x64 platform: x64

hoyosjs commented 5 years ago

@dotnet/dotnet-diag

jorive commented 5 years ago

/cc @sywhang @josalem @noahfalk

duskembayev commented 5 years ago

Are there any workarounds? I tried to set "EnableDiagnostics" environment variable, but it doesn't help.

hoyosjs commented 5 years ago

Is there any chance you can supply a sampling profiler trace of the process, say using ETW through PerfView? The shutdown path you're seeing there suggests there was a stack trace taken with no symbols, not that the shutdown is actually happening. Code review suggests that one thread is waiting on the call to Accept waiting for a connection, but I'd be surprised if that used this much CPU.

duskembayev commented 5 years ago

@hoyosjs I've collected profile data, but can not publish them here. Could you give me some email address to send it?

hoyosjs commented 5 years ago

Thanks @duskembayev

hoyosjs commented 4 years ago

An option to disable EventPipe's thread from spawning has been added in master and 3.0 through dotnet/coreclr#27137 and dotnet/coreclr#27140 respectively setting the environment variable COMPlus_EnableDiagnostics to 0 prior to launching the process. @duskembayev did you get a chance to test if this solves your problem? If not, please feel free to reopen this issue/send me an email.