search

StefanKert / BuildVision

A Visual Studio extension to visualize the building process.
MIT License
261 stars 43 forks source link

BuildVision causing ThreadPool thread leaking #106

Closed davkean closed 3 years ago

davkean commented 3 years ago

Hey, I'm from the Visual Studio performance team @ Microsoft.

We automatically capture data from Visual Studio instances with runaway thread and memory leaks, and BuildVision is showing up a top leak due to the following call stack:

    Microsoft.VisualStudio.Shell.14.0.dll!Microsoft.VisualStudio.Shell.ServiceProvider.QueryService(System.Guid guid, System.Type serviceType, bool setShellErrorInfo, out object service)  Unknown
    Microsoft.VisualStudio.Shell.14.0.dll!Microsoft.VisualStudio.Shell.ServiceProvider.QueryService(System.Type serviceType, bool setShellErrorInfo, out object service)    Unknown
>   Microsoft.VisualStudio.Shell.14.0.dll!Microsoft.VisualStudio.Shell.ServiceProvider.GetService(System.Type serviceType, bool setShellErrorInfo)  Unknown
    Microsoft.VisualStudio.Shell.14.0.dll!Microsoft.VisualStudio.Shell.ServiceProvider.GetService(System.Type serviceType)  Unknown
    BuildVision.dll!BuildVision.Extensions.IServiceProviderExtensions.GetService<Microsoft.VisualStudio.Shell.Interop.IVsStatusbar>(System.IServiceProvider serviceProvider)    Unknown
    BuildVision.dll!BuildVision.Core.StatusBarNotificationService.SetText(string str, bool freezeOutput)    Unknown
    BuildVision.dll!BuildVision.Core.StatusBarNotificationService.ShowTextWithFreeze(string str)    Unknown
    BuildVision.dll!BuildVision.Core.BuildInformationProvider.BuildUpdate() Unknown
    BuildVision.dll!BuildVision.Core.BuildInformationProvider.BuildStarted.AnonymousMethod__26_0(object state)  Unknown
    mscorlib.dll!System.Threading.TimerQueueTimer.CallCallbackInContext(object state)   Unknown
    mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)   Unknown
    mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)   Unknown
    mscorlib.dll!System.Threading.TimerQueueTimer.CallCallback()    Unknown
    mscorlib.dll!System.Threading.TimerQueueTimer.Fire()    Unknown
    mscorlib.dll!System.Threading.TimerQueue.FireQueuedTimerCompletion(object state)    Unknown
    mscorlib.dll!System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()  Unknown
    mscorlib.dll!System.Threading.ThreadPoolWorkQueue.Dispatch()    Unknown
    mscorlib.dll!System.Threading._ThreadPoolWaitCallback.PerformWaitCallback() Unknown

Here we have a ThreadPool thread firing up based on a timer and trying perform a COM RPC to the UI thread to query a service. While the UI thread is busy, this thread will block waiting while the timer continues to fire, continually adding threads that are blocked. In some dumps we have, this stack is consuming over 700 threads, and sometimes up to 1600 threads. This leaks quite a lot of memory:

Failure Size 90th (MB) Size 50th (MB) AvgSize (MB)
Thd_ThreadStack_buildvision!BuildVision.Extensions.IServiceProviderExtensions.GetService[[System.__Canon,mscorlib]] 1765.8 816.84 988.02

This code should be changed to only ever have one of these events running at the same time and should not assume that any GetService or access to IVsXXX will respond immediately. We have more information about this situation here: https://github.com/microsoft/vs-threading/blob/main/doc/threadpool_starvation.md.

StefanKert commented 3 years ago

Hi @davkean,

thank you very much for bringing that up. I wasn´t aware of the huge memory consumption / issues that BuildVision is causing.

I´ll check the PR in the next hour :) Thanks a lot.

If there is anything else that you have noticed while debugging / perfchecking please let me know.