Capturing from hwnd results in Fail Fast Exception 0xc0000409 from ucrtbase.dll

[dotnetdan]

Using .NET 6 & AvaloniaUI on Windows 10.0.19041, I'm attempting to capture frames from another application's hwnd. My capture session runs for about 15 seconds before suddenly terminating and creating a WER. The exception code is 0xc0000409 which is a fail fast exception:

Faulting application name: GUIScreenCapture.exe, version: 1.0.0.0, time stamp: 0x6216d2c0
Faulting module name: ucrtbase.dll, version: 10.0.19041.789, time stamp: 0x2bd748bf
Exception code: 0xc0000409
Fault offset: 0x000000000007286e
Faulting process id: 0x8a40
Faulting application start time: 0x01d854696769c70a
Faulting application path: C:\Users\dling\Source\repos\EmguMatchTemplate\GUIScreenCapture\bin\Debug\net6.0-windows10.0.19041.0\GUIScreenCapture.exe
Faulting module path: C:\WINDOWS\System32\ucrtbase.dll
Report Id: 653925f0-bf2c-495b-a1a2-aad1ce045c72
Faulting package full name: 
Faulting package-relative application ID: 

This is how I am creating my capture session. The Direct3D helper methods are from the WPF sample, I only modified it to use the new COM marshaling API's starting in .net 5.

   public class MainWindowViewModel : ViewModelBase
    {
        // bound to UI textbox
        public string ProcessName { get; set; }

        private static readonly IDirect3DDevice _d3dDevice = Direct3D.CreateDevice();

        public void StartCapturing()
        {
            Process process = Process.GetProcessesByName(ProcessName).Single();
            IntPtr hwndMainWindow = process.MainWindowHandle;
            GraphicsCaptureItem graphicsCaptureItem = Direct3D.CreateItemForWindow(hwndMainWindow);
            GraphicsCaptureSession session;
            var dispatcher = CoreMessagingHelper.CreateDispatcherQueueControllerForCurrentThread();
            Direct3D11CaptureFramePool framePool = Direct3D11CaptureFramePool.Create(
                _d3dDevice,
                DirectXPixelFormat.B8G8R8A8UIntNormalized,
                3,
                graphicsCaptureItem.Size);

            session = framePool.CreateCaptureSession(graphicsCaptureItem);
            framePool.FrameArrived += OnFrameArrived;
            session.StartCapture();
        }

        private static void OnFrameArrived(Direct3D11CaptureFramePool sender, object args)
        {
            using var frame = sender.TryGetNextFrame();
            if (frame != null)
            {
                Debug.WriteLine($"{frame.SystemRelativeTime} frame captured");
            }
            else
            {
                Debug.WriteLine("null frame");
            }
        }
    }

!analyze -v in WinDbg shows:

*******************************************************************************
*                                                                             *
*                        Exception Analysis                                   *
*                                                                             *
*******************************************************************************

Failed to request MethodData, not in JIT code range
MethodDesc:   00007ffb2733d320
Method Name:          WinRT.IObjectReference.Release()
Class:                00007ffb27349d20
MethodTable:          00007ffb2733d400
mdToken:              000000000600057D
Module:               00007ffb26ec2978
IsJitted:             yes
Current CodeAddr:     00007ffb2714c610
Version History:
MethodDesc:   00007ffb2733d2d8
Method Name:          WinRT.IObjectReference.Dispose(Boolean)
Class:                00007ffb27349d20
MethodTable:          00007ffb2733d400
mdToken:              0000000006000579
Module:               00007ffb26ec2978
IsJitted:             yes
Current CodeAddr:     00007ffb2714c4f0
Version History:
MethodDesc:   00007ffb2733d160
Method Name:          WinRT.IObjectReference.Finalize()
Class:                00007ffb27349d20
MethodTable:          00007ffb2733d400
mdToken:              000000000600056D
Module:               00007ffb26ec2978
IsJitted:             yes
Current CodeAddr:     00007ffb27184110
Version History:
Failed to request MethodData, not in JIT code range
MethodDesc:   00007ffb2733d320
Method Name:          WinRT.IObjectReference.Release()
Class:                00007ffb27349d20
MethodTable:          00007ffb2733d400
mdToken:              000000000600057D
Module:               00007ffb26ec2978
IsJitted:             yes
Current CodeAddr:     00007ffb2714c610
Version History:
MethodDesc:   00007ffb2733d2d8
Method Name:          WinRT.IObjectReference.Dispose(Boolean)
Class:                00007ffb27349d20
MethodTable:          00007ffb2733d400
mdToken:              0000000006000579
Module:               00007ffb26ec2978
IsJitted:             yes
Current CodeAddr:     00007ffb2714c4f0
Version History:
MethodDesc:   00007ffb2733d160
Method Name:          WinRT.IObjectReference.Finalize()
Class:                00007ffb27349d20
MethodTable:          00007ffb2733d400
mdToken:              000000000600056D
Module:               00007ffb26ec2978
IsJitted:             yes
Current CodeAddr:     00007ffb27184110
Version History:

KEY_VALUES_STRING: 1

    Key  : Analysis.CPU.mSec
    Value: 4843

    Key  : Analysis.DebugAnalysisManager
    Value: Create

    Key  : Analysis.Elapsed.mSec
    Value: 4900

    Key  : Analysis.Init.CPU.mSec
    Value: 12171

    Key  : Analysis.Init.Elapsed.mSec
    Value: 4772204

    Key  : Analysis.Memory.CommitPeak.Mb
    Value: 456

    Key  : CLR.Engine
    Value: CORECLR

    Key  : CLR.Version
    Value: 6.0.322.12309

    Key  : FailFast.Name
    Value: FATAL_APP_EXIT

    Key  : FailFast.Type
    Value: 7

    Key  : Timeline.Process.Start.DeltaSec
    Value: 11

    Key  : WER.OS.Branch
    Value: vb_release

    Key  : WER.OS.Timestamp
    Value: 2019-12-06T14:06:00Z

    Key  : WER.OS.Version
    Value: 10.0.19041.1

    Key  : WER.Process.Version
    Value: 1.0.0.0

FILE_IN_CAB:  GUIScreenCapture.exe.26416.dmp

NTGLOBALFLAG:  0

PROCESS_BAM_CURRENT_THROTTLED: 0

PROCESS_BAM_PREVIOUS_THROTTLED: 0

APPLICATION_VERIFIER_FLAGS:  0

CONTEXT:  00000048b1eff1c0 -- (.cxr 0x48b1eff1c0)
rax=0000000000000000 rbx=0000000000000000 rcx=0000000000000000
rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
rip=0000000000000000 rsp=0000000000000000 rbp=0000000000000000
 r8=0000000000000000  r9=0000000000000000 r10=0000000000000000
r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0         nv up di pl nz na pe nc
cs=ffff  ss=0000  ds=ffff  es=ffff  fs=ffff  gs=0000             efl=00000000
00000000`00000000 ??              ???
Resetting default scope

EXCEPTION_RECORD:  (.exr -1)
ExceptionAddress: 00007ffc4132286e (ucrtbase!abort+0x000000000000004e)
   ExceptionCode: c0000409 (Security check failure or stack buffer overrun)
  ExceptionFlags: 00000001
NumberParameters: 1
   Parameter[0]: 0000000000000007
Subcode: 0x7 FAST_FAIL_FATAL_APP_EXIT 

PROCESS_NAME:  GUIScreenCapture.exe

ERROR_CODE: (NTSTATUS) 0xc0000409 - The system detected an overrun of a stack-based buffer in this application. This overrun could potentially allow a malicious user to gain control of this application.

EXCEPTION_CODE_STR:  c0000409

EXCEPTION_PARAMETER1:  0000000000000007

FAULTING_THREAD:  00009be0

STACK_TEXT:  
00000048`b1efe010 00007ffc`41321f9f     : 00000048`00000003 00000048`00000003 00007ffc`412f9ee6 00000048`b1efe150 : ucrtbase!abort+0x4e
00000048`b1efe040 00007ffc`412eee66     : 00000048`b1eff320 00000048`b1efe910 00000048`b1eff320 00000048`b1efe320 : ucrtbase!terminate+0x1f
00000048`b1efe070 00007ffc`412efae4     : 00000000`00000000 00007ffc`4102986b 00000000`00000001 00000048`b1eff320 : ucrtbase!FindHandler<__FrameHandler4>+0x50a
00000048`b1efe240 00007ffc`412ed950     : 00007ffc`23120000 00000048`b1eff320 00000048`b1efead0 00000048`b1efe910 : ucrtbase!__InternalCxxFrameHandler<__FrameHandler4>+0x278
00000048`b1efe2e0 00007ffc`437120cf     : 00000048`b1eff500 00000000`00000000 00000048`b1efe8c0 00000048`b1eff320 : ucrtbase!_CxxFrameHandler4+0xa0
00000048`b1efe350 00007ffc`436c1454     : 00000000`00000000 00000048`b1efe8c0 00000048`b1eff320 00000000`00000001 : ntdll!RtlpExecuteHandlerForException+0xf
00000048`b1efe380 00007ffc`436c11a5     : 00000000`00000000 00000048`b1eff1c0 00000000`00000000 00000048`b1efead0 : ntdll!RtlDispatchException+0x244
00000048`b1efea90 00007ffc`41014f69     : 00000000`00000000 00007ffc`2313e0e0 00000000`00000001 00007ffc`430b82f2 : ntdll!RtlRaiseException+0x185
00000048`b1eff300 00007ffc`412eda1d     : 00000048`b1eff490 00007ffc`23123f92 00000000`00000000 00000000`00000000 : KERNELBASE!RaiseException+0x69
00000048`b1eff3e0 00007ffc`2312407a     : 00007ffc`23120000 00000202`7269ab20 00000202`7269ab20 00000048`b1eff660 : ucrtbase!CxxThrowException+0xad
00000048`b1eff450 00007ffc`23132fd6     : 00000202`6ba93a58 00000000`00000001 00002313`b69013c1 00000048`b1eff4b8 : GraphicsCapture!winrt::throw_hresult+0x6e
00000048`b1eff4a0 00007ffc`2312af90     : 00000fff`70ac83b4 00000048`b1eff490 00000048`b1eff4c0 00000048`b1eff580 : GraphicsCapture!winrt::Windows::Graphics::Capture::implementation::ServerCaptureSessionCore::StopCapture+0x26
00000048`b1eff4d0 00007ffc`2312ac04     : 00000202`00249ac0 00000202`7269ab20 00000048`b1eff660 00007ffb`2717adf0 : GraphicsCapture!winrt::Windows::Graphics::Capture::implementation::GraphicsCaptureSession::Close+0x34
00000048`b1eff500 00007ffc`2312aca4     : 00000000`00000001 00000000`00000000 00000202`00249ac0 00007ffb`273a8728 : GraphicsCapture!winrt::Windows::Graphics::Capture::implementation::GraphicsCaptureSession::~GraphicsCaptureSession+0x1c
00000048`b1eff530 00007ffc`2312b5c8     : 00000202`7269ab20 00000000`00000000 00000202`61c297e0 00000202`00249ac0 : GraphicsCapture!winrt::impl::heap_implements<winrt::Windows::Graphics::Capture::implementation::GraphicsCaptureSession>::`vector deleting destructor'+0x14
00000048`b1eff560 00007ffb`2714c6bb     : 00000000`00000000 00000048`b1eff550 00000000`00000000 00000048`b1eff660 : GraphicsCapture!winrt::implements<winrt::Windows::Graphics::Capture::implementation::GraphicsCaptureSession,winrt::Windows::Graphics::Capture::GraphicsCaptureSession,winrt::Windows::Graphics::Capture::IGraphicsCaptureSession2,winrt::Windows::Foundation::IClosable>::Release+0x64
00000048`b1eff590 00007ffb`2714c55a     : 00000202`00249ac0 00007ffb`27184163 00007ffb`26fb6070 00007ffb`2675ef51 : 0x00007ffb`2714c6bb
00000048`b1eff670 00007ffb`27184136     : 00000202`00249ac0 00000000`00000000 00000000`00000000 00000000`00000008 : 0x00007ffb`2714c55a
00000048`b1eff6c0 00007ffb`8566a646     : 00000202`00249ac0 00007ffb`266fada0 00000202`61c297e0 00000000`00000001 : 0x00007ffb`27184136
00000048`b1eff700 00007ffb`855900c4     : 00000000`00000000 00007ffb`855a49e5 0000235b`121fdd34 00000000`00000000 : coreclr!FastCallFinalizeWorker+0x6
00000048`b1eff730 00007ffb`8558fdb3     : 00000202`00249ac0 00000048`b1eff870 00007ffb`273a8728 00007ffb`273a8728 : coreclr!MethodTable::CallFinalizer+0x80
00000048`b1eff770 00007ffb`855b9198     : 00000000`00000001 00000000`00000001 00000000`00000130 00000000`00000000 : coreclr!FinalizerThread::FinalizeAllObjects+0xe7
00000048`b1eff8a0 00007ffb`855b8615     : 00000202`60268e50 00000000`00000000 00000000`00000000 00007ffb`855b8ace : coreclr!FinalizerThread::FinalizerThreadWorker+0x98
00000048`b1eff8d0 00007ffb`855b851a     : 00000202`60207400 00007ffb`8555eee3 00000000`00000000 00000000`00000000 : coreclr!ManagedThreadBase_DispatchMiddle+0x85
00000048`b1eff9b0 00007ffb`856249a1     : 00000000`00000001 ffffffff`ffffffff 00000202`60268e50 00000000`00000000 : coreclr!ManagedThreadBase_DispatchOuter+0xae
00000048`b1effa50 00007ffc`41cf7034     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : coreclr!FinalizerThread::FinalizerThreadStart+0x91
00000048`b1effb60 00007ffc`436c2651     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0x14
00000048`b1effb90 00000000`00000000     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x21

STACK_COMMAND:  ~6s ; .cxr ; kb

SYMBOL_NAME:  ucrtbase!abort+4e

MODULE_NAME: ucrtbase

IMAGE_NAME:  ucrtbase.dll

FAILURE_BUCKET_ID:  FAIL_FAST_FATAL_APP_EXIT_c0000409_ucrtbase.dll!abort

OS_VERSION:  10.0.19041.1

BUILDLAB_STR:  vb_release

OSPLATFORM_TYPE:  x64

OSNAME:  Windows 10

IMAGE_VERSION:  10.0.19041.789

FAILURE_ID_HASH:  {e31753ac-c98a-8055-3663-47e707543d20}

Followup:     MachineOwner
---------

[robmikh]

You need to keep the references to the objects to keep them alive. It's working for a few seconds because the garbage collector hasn't kicked in yet.

Make the GraphicsCaptureSession and Direct3D11CaptureFramePool member variables and that should solve your problem.

The fail fast itself is a different issue in that build, where a COM proxy is being poked on the wrong thread (perhaps the finalizer thread). I would make sure you call Dispose on both objects from the thread that created them when you're finished.

[dotnetdan]

That did the trick, thank you!