dotnet / diagnostics

This repository contains the source code for various .NET Core runtime diagnostic tools and documents.
MIT License
1.18k stars 354 forks source link

[dotnet-trace] Does not collect any call stacks #976

Closed VladimirKhil closed 4 years ago

VladimirKhil commented 4 years ago

I have installed and ran dotnet-trace 3.1.57502 on Ubuntu 18.04.2 LTS attaching to my ASP.NET Core 3.1 app running with systemctl. When opening collected trace data in PerfView I see only CPU_TIME and UNMANAGED_CODE_TIME records and no managed call stacks.

I tried to use different providers and flags but have no luck.

What am I doing wrong? Should I somehow modify my application to get correct data?

Thank you!

ybhpQ

josalem commented 4 years ago

@VladimirKhil, how did you invoke dotnet-trace and how did you end the trace? This looks to me like your trace is missing Rundown events and therefore doesn't have any jitted symbols. This can happen in cases where the target application or the tracer is not shut down cleanly.

VladimirKhil commented 4 years ago

@VladimirKhil, how did you invoke dotnet-trace and how did you end the trace? This looks to me like your trace is missing Rundown events and therefore doesn't have any jitted symbols. This can happen in cases where the target application or the tracer is not shut down cleanly.

@josalem I invoked it via SSH by command "dotnet-trace collect -p ". And yes, I could not stop it normally via <Ctrl+C> or . It just ignores my commands and continued capturing data. So I stopped it with <Ctrl+Z>. Do you it this is the source of the problem? Should I wait some more time for normal close command to complete?

josalem commented 4 years ago

There is data that is captured after pressing <ctrl+c> called Rundown that includes, among other things, the jitted symbols. My hunch is that you didn't get any of those by closing the program with <ctrl+z> (I don't recall off the top of my head what signal gets sent to program with that, SIGHUP?). If possible, wait after pressing <ctrl+c> and it should finish. If you are still unable to stop the trace, it might be due to how the SIGINT is being trafficked over SSH. In that case, you can use the undocumented --duration flag to trace for a set amount of time, e.g., dotnet-trace collect -p <pid> --duration 00:00:00:30 for 30 seconds.

VladimirKhil commented 4 years ago

--duration flag works! Thank you very much. It works for 2 minutes 15 seconds instead of 30 seconds but thats not a big problem - it finishes normally and I get all stack traces.

Thank you again!

josalem commented 4 years ago

Depending on the size of your target application, compute power of your environment, and I/O throughput, Rundown can take many seconds. It sounds like your Rundown took a long time. I'm glad the duration flag worked for you 😄

iliamosko commented 4 years ago

Is there any documentation about what is _UNMANAGED_CODETIME? When I look into the Thread Time stack, _UNMANAGED_CODETIME always takes up 100% of the time. It seems very obscure as to what exactly it is collecting.