renderdoc: backend-agnostic RenderDoc frame capture functionality

gfx-rs / gfx

[maintenance mode] A low-overhead Vulkan-like GPU API for Rust.

http://gfx-rs.github.io/

Apache License 2.0

5.35k stars 551 forks source link

renderdoc: backend-agnostic RenderDoc frame capture functionality #3748

Closed manuel-woelker closed 3 years ago

manuel-woelker commented 3 years ago

This is a follow up to #3737 to bring RenderDoc frame capture functionality to more backends

This is a first implementation sketch to bring RenderDoc frame capture to more backends. My first goal was to get the internal API right, and figure out if gfx-hal is the right location for this code (maybe auxil would be better suited?)

Fixes #issue PR checklist:

[x] make succeeds ~~(on *nix)~~ on Windows 10
[x] make reftests succeeds
[x] tested examples with the following backends: Vulkan, DX11, DX12

kvark commented 3 years ago

This shouldn't be in gfx-hal because it's not a part of the API. It's just a shared implementation block. If you think it's worth sharing, let's move it into a separate crate, like src/auxil/renderdoc?

manuel-woelker commented 3 years ago

I moved the renderdoc code to a new crate gfx-auxil-renderdoc and used that in the Vulkan and DX12 backends.

The DX12 implementation seems to work, but RenderDoc crashes when loading the capture, not sure what the issue is there, will investigate further.

manuel-woelker commented 3 years ago

Here's my test results so far

Vulkan: works fine
DX12: seems to work, but RenderDoc still crashes when I load the capture with the error log mentioning RDOC 002736: [19:55:52] d3d12_device.cpp(3365) - Error - Assertion failed: '(hr) == (((HRESULT)0L))' (hr=DXGI_ERROR_DEVICE_HUNG, ((HRESULT)0L)=S_OK) and RDOC 002736: [19:55:52] d3d12_command_list_wrap.cpp( 271) - Error - Failed on resource serialise-creation, hr: DXGI_ERROR_DEVICE_REMOVED, not sure what's going on there
DX11: the compute example won't run because there doesn't seem to be a compatible memory type: thread 'main' panicked at 'calledOption::unwrap()on aNonevalue', examples\compute/main.rs:304:10

kvark commented 3 years ago

Looks like for D3D12 we'd need to try reproducing this in a smaller example. I.e. something using d3d12-rs and winapi directly.

manuel-woelker commented 3 years ago

After some fruitless dead-ends I decided to try to use the quad example again to verify the renderdoc integration. This fixed both of the issues I had. Captures now look good on Vulkan, DX11 and DX12.

kvark commented 3 years ago

After some fruitless dead-ends I decided to try to use the quad example again to verify the renderdoc integration.

So what was the problem then, originally?

bors[bot] commented 3 years ago

Build succeeded: