dotnet / wpf

WPF is a .NET Core UI framework for building Windows desktop applications.
MIT License
7.07k stars 1.17k forks source link

OutOfMemoryException from DUCE.Channel.SyncFlush() #8397

Open haefele opened 11 months ago

haefele commented 11 months ago

Description

One of our customers, is getting a weird OutOfMemoryException deep inside the framework code, even tho not much is happening in our app.

I've found these two other issues that seem similar: #7378 #8031

Reproduction Steps

Sadly we can't reproduce the problem on any of our own machines.

But our customer is able to reliable reproduce the problem. We gathered some process-dumps during some crashes, and that's how we figured out everything we know so far.

We could maybe share those process-dumps with you.

Expected behavior

Not getting an OutOfMemoryException, the application should keep working as usual.

Actual behavior

The exception:

System.OutOfMemoryException: Insufficient memory to continue the execution of the program.
   at System.Windows.Media.Composition.DUCE.Channel.SyncFlush()
   at System.Windows.Interop.HwndTarget.UpdateWindowSettings(Boolean enableRenderTarget, Nullable`1 channelSet)
   at System.Windows.Interop.HwndTarget.UpdateWindowPos(IntPtr lParam)
   at System.Windows.Interop.HwndTarget.HandleMessage(WindowMessage msg, IntPtr wparam, IntPtr lparam)
   at System.Windows.Interop.HwndSource.HwndTargetFilterMessage(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled) in MS.Win32\HwndWrapper.cs:line 192
   at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o) in MS.Win32\HwndSubclass.cs:line 223
   at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs) in System.Windows.Threading\ExceptionWrapper.cs:line 73
   at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler) in System.Windows.Threading\ExceptionWrapper.cs:line 24

It looks like this problem arises when a tooltip pops up in the app. I receive the System.OutOfMemoryException within the Dispatcher.UnhandledException event, with the following stacktrace.

However, it also could be unrelated.

    OurApp.dll!OurApp.WPF.UI.App.Application_DispatcherUnhandledException(object sender, System.Windows.Threading.DispatcherUnhandledExceptionEventArgs e) Line 215 C#
    WindowsBase.dll!System.Windows.Threading.Dispatcher.CatchException(System.Exception e) Line 1705    C#
    WindowsBase.dll!System.Windows.Threading.Dispatcher.CatchExceptionStatic(object source, System.Exception e) Line 1692   C#
    WindowsBase.dll!System.Windows.Threading.ExceptionWrapper.CatchException(object source, System.Exception e, System.Delegate catchHandler) Line 116  C#
    WindowsBase.dll!System.Windows.Threading.ExceptionWrapper.TryCatchWhen(object source, System.Delegate callback, object args, int numArgs, System.Delegate catchHandler) Line 28 C#
    WindowsBase.dll!System.Windows.Threading.Dispatcher.LegacyInvokeImpl(System.Windows.Threading.DispatcherPriority priority, System.TimeSpan timeout, System.Delegate method, object args, int numArgs) Line 981  C#
    WindowsBase.dll!MS.Win32.HwndSubclass.SubclassWndProc(System.IntPtr hwnd, int msg, System.IntPtr wParam, System.IntPtr lParam) Line 180 C#
    [Native to Managed Transition]  
    [Managed to Native Transition]  
    PresentationCore.dll!System.Windows.Interop.HwndSource.Resize(System.Windows.Size newSize)  Unknown
    PresentationCore.dll!System.Windows.Interop.HwndSource.OnLayoutUpdated(object obj, System.EventArgs args)   Unknown
    PresentationCore.dll!System.Windows.ContextLayoutManager.fireLayoutUpdateEvent()    Unknown
    PresentationCore.dll!System.Windows.ContextLayoutManager.UpdateLayout() Unknown
    PresentationCore.dll!System.Windows.Interop.HwndSource.SetLayoutSize()  Unknown
    PresentationCore.dll!System.Windows.Interop.HwndSource.RootVisualInternal.set(System.Windows.Media.Visual value)    Unknown
    PresentationCore.dll!System.Windows.Interop.HwndSource.RootVisual.set(System.Windows.Media.Visual value)    Unknown
    PresentationFramework.dll!System.Windows.Controls.Primitives.Popup.SetRootVisualToPopupRoot()   Unknown
    PresentationFramework.dll!System.Windows.Controls.Primitives.Popup.CreateWindow(bool asyncCall) Unknown
    PresentationFramework.dll!System.Windows.Controls.Primitives.Popup.OnIsOpenChanged(System.Windows.DependencyObject d, System.Windows.DependencyPropertyChangedEventArgs e)  Unknown
    WindowsBase.dll!System.Windows.DependencyObject.OnPropertyChanged(System.Windows.DependencyPropertyChangedEventArgs e) Line 1354    C#
    PresentationFramework.dll!System.Windows.FrameworkElement.OnPropertyChanged(System.Windows.DependencyPropertyChangedEventArgs e)    Unknown
    WindowsBase.dll!System.Windows.DependencyObject.NotifyPropertyChange(System.Windows.DependencyPropertyChangedEventArgs args) Line 1185  C#
    WindowsBase.dll!System.Windows.DependencyObject.UpdateEffectiveValue(System.Windows.EntryIndex entryIndex, System.Windows.DependencyProperty dp, System.Windows.PropertyMetadata metadata, System.Windows.EffectiveValueEntry oldEntry, ref System.Windows.EffectiveValueEntry newEntry, bool coerceWithDeferredReference, bool coerceWithCurrentValue, System.Windows.OperationType operationType) Line 1133   C#
    WindowsBase.dll!System.Windows.DependencyObject.SetValueCommon(System.Windows.DependencyProperty dp, object value, System.Windows.PropertyMetadata metadata, bool coerceWithDeferredReference, bool coerceWithCurrentValue, System.Windows.OperationType operationType, bool isInternal) Line 811   C#
    WindowsBase.dll!System.Windows.DependencyObject.SetValue(System.Windows.DependencyProperty dp, object value) Line 604   C#
    PresentationFramework.dll!System.Windows.Data.BindingOperations.SetBinding(System.Windows.DependencyObject target, System.Windows.DependencyProperty dp, System.Windows.Data.BindingBase binding)   Unknown
    PresentationFramework.dll!System.Windows.Controls.Primitives.Popup.CreateRootPopupInternal(System.Windows.Controls.Primitives.Popup popup, System.Windows.UIElement child, bool bindTreatMousePlacementAsBottomProperty)    Unknown
    PresentationFramework.dll!System.Windows.Controls.ToolTip.HookupParentPopup()   Unknown
    PresentationFramework.dll!System.Windows.Controls.ToolTip.OnIsOpenChanged(System.Windows.DependencyObject d, System.Windows.DependencyPropertyChangedEventArgs e)   Unknown
    WindowsBase.dll!System.Windows.DependencyObject.OnPropertyChanged(System.Windows.DependencyPropertyChangedEventArgs e) Line 1354    C#
    PresentationFramework.dll!System.Windows.FrameworkElement.OnPropertyChanged(System.Windows.DependencyPropertyChangedEventArgs e)    Unknown
    WindowsBase.dll!System.Windows.DependencyObject.NotifyPropertyChange(System.Windows.DependencyPropertyChangedEventArgs args) Line 1185  C#
    WindowsBase.dll!System.Windows.DependencyObject.UpdateEffectiveValue(System.Windows.EntryIndex entryIndex, System.Windows.DependencyProperty dp, System.Windows.PropertyMetadata metadata, System.Windows.EffectiveValueEntry oldEntry, ref System.Windows.EffectiveValueEntry newEntry, bool coerceWithDeferredReference, bool coerceWithCurrentValue, System.Windows.OperationType operationType) Line 1133   C#
    WindowsBase.dll!System.Windows.DependencyObject.SetValueCommon(System.Windows.DependencyProperty dp, object value, System.Windows.PropertyMetadata metadata, bool coerceWithDeferredReference, bool coerceWithCurrentValue, System.Windows.OperationType operationType, bool isInternal) Line 811   C#
    WindowsBase.dll!System.Windows.DependencyObject.SetValue(System.Windows.DependencyProperty dp, object value) Line 604   C#
    PresentationFramework.dll!System.Windows.Controls.PopupControlService.ShowToolTip(System.Windows.DependencyObject o, bool fromKeyboard) Unknown
    PresentationFramework.dll!System.Windows.Controls.PopupControlService.PromotePendingToolTipToCurrent(System.Windows.Controls.ToolTipService.TriggerAction triggerAction)    Unknown
>>>>    PresentationFramework.dll!System.Windows.Controls.PopupControlService.BeginShowToolTip.AnonymousMethod__0(object s, System.EventArgs e) Unknown
    WindowsBase.dll!System.Windows.Threading.DispatcherTimer.FireTick() Line 254    C#
    WindowsBase.dll!System.Windows.Threading.ExceptionWrapper.InternalRealCall(System.Delegate callback, object args, int numArgs) Line 73  C#
    WindowsBase.dll!System.Windows.Threading.ExceptionWrapper.TryCatchWhen(object source, System.Delegate callback, object args, int numArgs, System.Delegate catchHandler) Line 24 C#
    WindowsBase.dll!System.Windows.Threading.DispatcherOperation.InvokeImpl() Line 424  C#
    WindowsBase.dll!MS.Internal.CulturePreservingExecutionContext.CallbackWrapper(object obj) Line 100  C#
    System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)   Unknown
    WindowsBase.dll!MS.Internal.CulturePreservingExecutionContext.Run(MS.Internal.CulturePreservingExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 85  C#
    WindowsBase.dll!System.Windows.Threading.DispatcherOperation.Invoke() Line 351  C#
    WindowsBase.dll!System.Windows.Threading.Dispatcher.ProcessQueue() Line 1243    C#
    WindowsBase.dll!System.Windows.Threading.Dispatcher.WndProcHook(System.IntPtr hwnd, int msg, System.IntPtr wParam, System.IntPtr lParam, ref bool handled) Line 1359    C#
    WindowsBase.dll!MS.Win32.HwndWrapper.WndProc(System.IntPtr hwnd, int msg, System.IntPtr wParam, System.IntPtr lParam, ref bool handled) Line 192    C#
    WindowsBase.dll!MS.Win32.HwndSubclass.DispatcherCallbackOperation(object o) Line 223    C#
    WindowsBase.dll!System.Windows.Threading.ExceptionWrapper.InternalRealCall(System.Delegate callback, object args, int numArgs) Line 73  C#
    WindowsBase.dll!System.Windows.Threading.ExceptionWrapper.TryCatchWhen(object source, System.Delegate callback, object args, int numArgs, System.Delegate catchHandler) Line 24 C#
    WindowsBase.dll!System.Windows.Threading.Dispatcher.LegacyInvokeImpl(System.Windows.Threading.DispatcherPriority priority, System.TimeSpan timeout, System.Delegate method, object args, int numArgs) Line 981  C#
    WindowsBase.dll!MS.Win32.HwndSubclass.SubclassWndProc(System.IntPtr hwnd, int msg, System.IntPtr wParam, System.IntPtr lParam) Line 180 C#
    [Native to Managed Transition]  
    [Managed to Native Transition]  
    WindowsBase.dll!System.Windows.Threading.Dispatcher.PushFrameImpl(System.Windows.Threading.DispatcherFrame frame) Line 1273 C#
    WindowsBase.dll!System.Windows.Threading.Dispatcher.PushFrame(System.Windows.Threading.DispatcherFrame frame) Line 459  C#
    WindowsBase.dll!System.Windows.Threading.Dispatcher.Run() Line 430  C#
    PresentationFramework.dll!System.Windows.Application.RunDispatcher(object ignore)   Unknown
    PresentationFramework.dll!System.Windows.Application.RunInternal(System.Windows.Window window)  Unknown
    PresentationFramework.dll!System.Windows.Application.Run()  Unknown
    OurApp.dll!OurApp.WPF.UI.App.Main() C#

Regression?

We don't know if this is a recent problem or if it's always been there for the customer. Neither do we know if just a single machine or windows-user is affected by it.

But we can pretty confidently say, that the issue doesn't seem very common, as our app doesn't show this issue on thousands of other machines.

Known Workarounds

No response

Impact

Our app crashes, and with it all unsaved changes are lost.

Configuration

Other information

No response

lindexi commented 11 months ago

In most cases, the application is out of memory.

cuiliang commented 11 months ago

Have same issue in net472 application, so many reports from different users, different os versions from win7 to win11.

OutOfMemoryException: 没有足够的内存继续执行程序。
StackTrace:
   在 System.Windows.Media.Composition.DUCE.Channel.SyncFlush()
   在 System.Windows.Interop.HwndTarget.UpdateWindowSettings(Boolean enableRenderTarget, Nullable`1 channelSet)
   在 System.Windows.Interop.HwndTarget.UpdateWindowPos(IntPtr lParam)
   在 System.Windows.Interop.HwndTarget.HandleMessage(WindowMessage msg, IntPtr wparam, IntPtr lparam)
   在 System.Windows.Interop.HwndSource.HwndTargetFilterMessage(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   在 MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   在 MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
   在 System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
   在 System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)

One of the sample report, when this happen the memory status: Free system memory:10240MB Total System Memory:16269MB Current Process: WorkingSet:93MB PrivateMemory:275MB VirtualMemorySize:5380MB

So it maybe not released to shortage of memory.

cuiliang commented 11 months ago

@haefele Are you using any 3rd wpf library in your application?

inosik commented 11 months ago

Note that stack traces of OutOfMemoryExceptions don't necessarily point to the problematic code.

At my company we had an OOM in our app with a similar stack trace as shown here. The actual problem was that we used an overload of the XmlSerializer constructor which generates an in-memory assembly every time it is called.

haefele commented 11 months ago

@haefele Are you using any 3rd wpf library in your application?

Yes, we are making heavy use of DevExpress WPF controls.

Note that stack traces of OutOfMemoryExceptions don't necessarily point to the problematic code.

At my company we had an OOM in our app with a similar stack trace as shown here. The actual problem was that we used an overload of the XmlSerializer constructor which generates an in-memory assembly every time it is called.

That's interesting. How did you figure that out in the end?

I still have the process-dumps at hand, and usually am pretty okay figuring stuff out like that, but I really don't see anything suspicious in those.


I received some more info from the customer. It seems like they experience that problem on just a single machine. Could it be hardware or driver related?

inosik commented 11 months ago

How did you figure that out in the end?

I'll spare you the boring details and the fact that it took us some weeks to open the crash dump in Visual Studio and just take a look at the Modules window 😅

I still have the process-dumps at hand, and usually am pretty okay figuring stuff out like that, but I really don't see anything suspicious in those.

You can try to analyze the dumps using the built-in tools in VS or try a memory profiler like dotMemory.

DK189 commented 1 day ago

I had the same issue but it was on DUCE::ReleaseOnChannel, I'm not sure what happened as I was leaving the application debugged in the background and then the IDE threw an error as soon as I focused on the MainWindow.

At that time, my computer was using 77% of the memory and my application was taking up ~358.5MB. Obviously there was plenty of free memory for the application to work. The same application when running before could sometimes use up to 1.2GB of memory when doing data collection.

When I took a memory snapshot, the most used things were GlyphRun, Hashtable, ResouceDictionary, RenderData,... while the data in my application model was negligible.

The application was running WPF on .NetFramework 4.7.2, the IDE was Visual Studio 2022 Professional 17.11.4