baldurk / renderdoc

RenderDoc is a stand-alone graphics debugging tool.
https://renderdoc.org
MIT License
8.62k stars 1.3k forks source link

FlushPendingDead flushes dead resources when capturing #3281

Closed bredbored closed 2 months ago

bredbored commented 3 months ago

This prevents crashing when IDXGISwapChain::ResizeBuffers occurs during the capture, which seems to happen frequently in Unreal Editor.

Description

Capturing D3D11 in Unreal Edtior crashed frequently for me. Eventually I investigated and found that frequently in my captures, a call to IDXGISwapChain::ResizeBuffers occurred. ResizeBuffers requires that there are no references to the swap chain textures, but I found that an active capture defers releasing resources until the capture is complete. If ResizeBuffers occurred during a capture, the resize failed. Unreal Editor detected the failure and intentionally crashed. I removed the deferment, and Unreal no longer crashes during my captures. I haven't noticed any negative consequences of removing the deferment.

baldurk commented 3 months ago

I'm not entirely sure this is safe, deferring destruction of resources until after the capture is complete is deliberate and since D3D11 is a fairly stable/unchanging target at this point I don't want to make such a large change like this without testing much wider than one application. Refcounting on D3D is extremely messy and it's easy for a solution to work in many cases and break in one specific edge case.

From what you describe the problem is limited to ResizeBuffers on swapchains where the refcount must hit 0, so I would rather you make a more targetted change to only exclude swapchain textures from the deferment (similar to contexts) and otherwise keep the code the same.

baldurk commented 2 months ago

Closing this due to lack of activity from the PR author and no response to feedback.

If you are the author and you'd like to revisit this change please open a new PR and address the feedback above.