Timeline profiling causes DotTrace.SaveData() to never return

[Vaccano]

I have a .Net Framework 4.x WPF application that occasionally (1% occurrence rate) has a performance issue when loading data. I have narrowed the problem down to a very specific part of the WPF application that is switching screens in my WPF application.

To try to get more information, I added dotTrace self profiling to my application. It starts up a profiling snapshot, then if the performance issue occurs, saves it out (if not, then the snapshot is discarded). 

This all works fine if I use the default "Sampling" style of profiling.  But if I call config.UseTimelineProfilingType(false); during the setup of the profiler, then I have problems.  Specifically, when I later call DotTrace.SaveData(); the call never returns.  If I call it from my UI thread then the application hangs.  If I call it from a background thread, then that thread never ends.  

This is compounded when I go to try to startup another snapshot.  The call to DotTrace.StartCollectingData() also never returns once the call to DotTrace.SaveData() has gone into an nonresponsive state.

I have tried just giving it a long time to save the file, but even after several hours, the file does not save.  (And it takes just a few seconds to successfully save when using the "Sampling" default style of profiling.)

I would love to be using the Timeline style of profiling.  Is there something I can do to make this work?

[Vaccano]

FYI: I am able to work around this issue by using the command line version of dotTrace. (It seems to support working with the Timeline style of profiling just fine.)

I use the api to download the command line version of dotTrace, then send commands to it via stdin (and parse stdout to know if it was successful). Definitely more of a pain than just using a nice API, but it works to get me up and running using Timeline profiling.

[gassanosh]

Thank you for creating the issue. We would like to investigate the case but unfortunately we cannot reproduce it and can do nothing without logs. Could you please collect the logs for us doing the following steps:

1. Delete old logs from those folders: %temp%\JetLogs and %systemroot%\Temp\JetLogs
2. Configure self api to write core logs: something like config.WithCommandLineArgument(@"--debug-logging --log-file=""<path-to-log-file>""") before you call DotTrace.Attach(config).
3. Reproduce the issue.
4. We need logs (all new files) from folders %temp%\JetLogs and %systemroot%\Temp\JetLogs along with clt logs you specified in step 2 (<path-to-log-file>).

Note: Log files can contain your personal information (your login name, folder names, etc.). You can securely attach those logs creating an issue in JetBrains issue tracker https://youtrack.jetbrains.com/issues/DTRC or submit request using https://dotnettools-support.jetbrains.com/hc/en-us/requests/new form.

[Vaccano]

I just went back to reproduce this issue and it did not fail.  I am not sure why it is working now.  

I had to reboot several times when setting up the command line interface (because dotTrace would eventually consume all the loggers and I could not figure out how to free them).  I am wondering if those reboots somehow fixed it.

Wish I would have know that would fix it.  I spent a few days on getting the command line connection up and running.

[gassanosh]

Usually you don't have to do anything extra, just call api. However If you want to profile using Timeline profiling type you should either make sure the ETW service ( https://www.jetbrains.com/profiler/etw/ ) is running or you should call UseTimelineProfilingType(true) and accept UAC. But even if you don't you just get an error, not hang. So your case is quite unusual. There is a chance you have some issues with ETW subsystem that were resolved after reboot.

[Vaccano]

@gassanosh - I was able to reproduce this issue. It occurs when the registry key Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\WMI\EtwMaxLoggers is either not set or not set high enough for the current amount of loggers on the machine (usually setting the DWORD to 128 (binary) is enough to make the error go away.).

I would recommend throwing an exception if it is possible to detect this.

For anyone else having this issue, this is the code that I have put in my application to at least know when this issue occurs:

var task = Task.Run((Action)DotTrace.SaveData);
task.Wait(10000);
if (!task.IsCompleted)
{
  // Here I notified the user that a fatal error had occurred and I shutdown the application.  I also indicate what should be done to deal with the issue for the next run.
}

[gassanosh]

Thank you for the response. Indeed self-api should throw exception if fatal error occurred. Probably exception should be thrown right after Attach but not after Save command.