CodeMaid is deadlocking Visual Studio

davkean commented 1 year ago

Environment

Visual Studio version: All versions of Visual Studio from 16.11 -> 17.4 Preview
CodeMaid version: 12.00.300.0
Code language: Unknown

Description

Our telemetry shows that the CodeMaid extension is hanging Visual Studio, causing folks to use TaskMgr to tear it down. Over the past 21 days, 17.3 has had (sampled) 124 hits, putting it at #90 hang overall in Visual Studio.

Steps to recreate

This is from telemetry data, there are no repro steps.

UI thread is blocked waiting on a lock:

    [Waiting on lock owned by Thread 26940, double-click or press enter to switch to thread]    
    ntdll.dll!_ZwWaitForMultipleObjects@20() Line 825   Unknown
>   KERNELBASE.dll!WaitForMultipleObjectsEx(unsigned long nCount, void * const * lpHandles, int bWaitAll, unsigned long dwMilliseconds, int bAlertable) Line 1551   C
    WindowsBase.ni.dll!511a4f49()   Unknown
    [Frames below may be incorrect and/or missing, native debugger attempting to walk managed call stack]   
    WindowsBase.ni.dll!511a4f49()   Unknown
    [Managed to Native Transition]  
    WindowsBase.dll!MS.Win32.UnsafeNativeMethods.WaitForMultipleObjectsEx(int nCount, System.IntPtr[] pHandles, bool bWaitAll, int dwMilliseconds, bool bAlertable) Unknown
    WindowsBase.dll!System.Windows.Threading.DispatcherSynchronizationContext.Wait(System.IntPtr[] waitHandles, bool waitAll, int millisecondsTimeout)  Unknown
    mscorlib.dll!System.Threading.SynchronizationContext.InvokeWaitMethodHelper(System.Threading.SynchronizationContext syncContext, System.IntPtr[] waitHandles, bool waitAll, int millisecondsTimeout)    Unknown
    [Native to Managed Transition]  
    [Managed to Native Transition]  
    mscorlib.dll!System.Lazy<EnvDTE.vsCMAccess>.LazyInitValue() Unknown
    mscorlib.dll!System.Lazy<EnvDTE.vsCMAccess>.Value.get() Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.UI.Converters.CodeItemToImageConverter.GetAccessString(SteveCadwallader.CodeMaid.Model.CodeItems.BaseCodeItemElement codeItem)  Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.UI.Converters.CodeItemToImageConverter.BuildImageURIString(SteveCadwallader.CodeMaid.Model.CodeItems.BaseCodeItem codeItem) Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.UI.Converters.CodeItemToImageConverter.Convert(object value, System.Type targetType, object parameter, System.Globalization.CultureInfo culture)    Unknown

Background thread is holding that lock (within Lazy<T>) and trying to switch to UI thread resulting in deadlock:

    ntdll.dll!_ZwWaitForMultipleObjects@20() Line 825   Unknown
    KERNELBASE.dll!WaitForMultipleObjectsEx(unsigned long nCount, void * const * lpHandles, int bWaitAll, unsigned long dwMilliseconds, int bAlertable) Line 1551   C
    mscorlib.ni.dll!733b42c8()  Unknown
    [Managed to Native Transition]  
    mscorlib.dll!System.Threading.Monitor.Wait(object obj, int millisecondsTimeout, bool exitContext)   Unknown
    mscorlib.dll!System.Threading.Monitor.Wait(object obj, int millisecondsTimeout) Unknown
    mscorlib.dll!System.Threading.ManualResetEventSlim.Wait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken)  Unknown
    mscorlib.dll!System.Threading.Tasks.Task.SpinThenBlockingWait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken)    Unknown
    mscorlib.dll!System.Threading.Tasks.Task.InternalWait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken)    Unknown
    mscorlib.dll!System.Threading.Tasks.Task.Wait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken)    Unknown
    mscorlib.dll!System.Threading.Tasks.Task.Wait(System.TimeSpan timeout)  Unknown
    [Waiting on Async Operation, double-click or press enter to view Async Call Stacks] 
    Microsoft.VisualStudio.Threading.dll!Microsoft.VisualStudio.Threading.JoinableTaskFactory.WaitSynchronouslyCore(System.Threading.Tasks.Task task)   Unknown
    Microsoft.VisualStudio.Threading.dll!Microsoft.VisualStudio.Threading.JoinableTaskFactory.WaitSynchronously(System.Threading.Tasks.Task task)   Unknown
    Microsoft.VisualStudio.Threading.dll!Microsoft.VisualStudio.Threading.JoinableTask.CompleteOnCurrentThread()    Unknown
>   SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Helpers.TextDocumentHelper.RunOnUIThread(System.Action action)  Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Helpers.TextDocumentHelper.GetTextToFirstMatch(EnvDTE.TextPoint startPoint, string matchString) Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Helpers.CodeElementHelper.GetMethodDeclaration(EnvDTE.CodeFunction codeFunction)    Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Helpers.ExplicitInterfaceImplementationHelper.IsExplicitInterfaceImplementation(EnvDTE80.CodeFunction2 codeFunction)    Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.CodeItemMethod..ctor.AnonymousMethod__6_6() Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.BaseCodeItemElement.TryDefault<bool>(System.Func<bool> func)    Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.BaseCodeItemElement.LazyTryDefault.AnonymousMethod__0() Unknown
    mscorlib.dll!System.Lazy<bool>.CreateValue()    Unknown
    mscorlib.dll!System.Lazy<bool>.LazyInitValue()  Unknown
    mscorlib.dll!System.Lazy<bool>.Value.get()  Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.CodeItemMethod..ctor.AnonymousMethod__6_0() Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.BaseCodeItemElement.TryDefault<EnvDTE.vsCMAccess>(System.Func<EnvDTE.vsCMAccess> func)  Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.BaseCodeItemElement.LazyTryDefault.AnonymousMethod__0() Unknown
    mscorlib.dll!System.Lazy<EnvDTE.vsCMAccess>.CreateValue()   Unknown
    mscorlib.dll!System.Lazy<EnvDTE.vsCMAccess>.LazyInitValue() Unknown
    mscorlib.dll!System.Lazy<EnvDTE.vsCMAccess>.Value.get() Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.BaseCodeItemElement.LoadLazyInitializedValues() Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.CodeItemMethod.LoadLazyInitializedValues()  Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeModelManager.LoadLazyInitializedValues(SteveCadwallader.CodeMaid.Model.CodeModel codeModel)   Unknown
    SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeModelManager.RetrieveAllCodeItemsAsync.AnonymousMethod__0()   Unknown

Current behavior

This code is taking a lock and then attempting to switch to the UI thread while holding that lock. This switch is blocked because the UI thread is currently blocked waiting on that lock.

Expected behavior

Change LazyThreadSafetyMode.Publication to allow both to run at the same time, which will prevent the UI thread from trying to take lock

-or-

Switch To AsyncLazy<T> passing in ThreadHelper.JoinableTaskFactory so that the UI thread will allow the background thread to proceed and run on the UI thread if its blocked on it

-or-

Switch to the UI thread before accessing the Lazy<T>

codecadwallader commented 1 year ago

Thanks for reporting the issue and providing the telemetry that's available. I have a proposed fix in #951 based on the first suggestion. However, I'm not confident that will address the underlying issue. It seems like this would be effective if there were two threads going into the same Lazy initializer but that doesn't appear to be the case from the telemetry. I may mis-understand how the LazyThreadSafetyMode affects other application code though?

davkean commented 6 months ago

Sorry Steve, I missed your reply to this.

It seems like this would be effective if there were two threads going into the same Lazy initializer but that doesn't appear to be the case from the telemetry.

That's the problem, we have two threads trying to get access to the Lazy initializer. When the UI thread wins and the background thread blocks, the product doesn't hang. However, when the background wins and UI thread blocks, then product hangs. This is because UI thread will not "pump" the message that the background thread needs to access the UI thread.

The fix should be good, I haven't seen an update on https://marketplace.visualstudio.com/items?itemName=SteveCadwallader.CodeMaidVS2022 with that version? This continues to rank high and has seen 173 hits past 30 days at a sampling of 10%, so the real numbers are probably close to 1730 hits.

codecadwallader / codemaid