search

baldurk / renderdoc

RenderDoc is a stand-alone graphics debugging tool.
https://renderdoc.org
MIT License
9.02k stars 1.35k forks source link

Crash when a multi-subpass `VkRenderPass` is created with `VkRenderPassInputAttachmentAspectCreateInfo` #2748

Closed glebov-andrey closed 2 years ago

glebov-andrey commented 2 years ago

Description

The RenderDoc application crashes while loading a capture containing a VkRenderPass with multiple passes which was created with a VkRenderPassInputAttachmentAspectCreateInfo structure in its VkRenderPassCreateInfo chain.

The stack trace: (click to expand) ``` nvoglv64.dll!00007ffa52ca8e45() nvoglv64.dll!00007ffa52ca7ffd() renderdoc.dll!WrappedVulkan::Serialise_vkCreateRenderPass(ReadSerialiser & ser, VkDevice_T *) Line 1119 at c:\build\renderdoc\renderdoc\driver\vulkan\wrappers\vk_misc_funcs.cpp(1119) renderdoc.dll!WrappedVulkan::ProcessChunk(ReadSerialiser & ser, VulkanChunk chunk) Line 3261 at c:\build\renderdoc\renderdoc\driver\vulkan\vk_core.cpp(3261) renderdoc.dll!WrappedVulkan::ReadLogInitialisation(RDCFile * rdc, bool storeStructuredBuffers) Line 2639 at c:\build\renderdoc\renderdoc\driver\vulkan\vk_core.cpp(2639) renderdoc.dll!VulkanReplay::ReadLogInitialisation(RDCFile * rdc, bool storeStructuredBuffers) Line 225 at c:\build\renderdoc\renderdoc\driver\vulkan\vk_replay.cpp(225) renderdoc.dll!ReplayController::PostCreateInit(IReplayDriver * device, RDCFile * rdc) Line 2148 at c:\build\renderdoc\renderdoc\replay\replay_controller.cpp(2148) renderdoc.dll!ReplayController::CreateDevice(RDCFile * rdc, const ReplayOptions & opts) Line 2115 at c:\build\renderdoc\renderdoc\replay\replay_controller.cpp(2115) renderdoc.dll!CaptureFile::OpenCapture(const ReplayOptions & opts, std::function progress) Line 371 at c:\build\renderdoc\renderdoc\replay\capture_file.cpp(371) qrenderdoc.exe!ReplayManager::run(int proxyRenderer, const QString & capturefile, const ReplayOptions & opts, std::function progress) Line 452 at c:\build\renderdoc\qrenderdoc\code\replaymanager.cpp(452) [Inline Frame] qrenderdoc.exe!ReplayManager::OpenCapture::__l2::::operator()() Line 58 at c:\build\renderdoc\qrenderdoc\code\replaymanager.cpp(58) [Inline Frame] qrenderdoc.exe!std::_Invoker_functor::_Call(ReplayManager::OpenCapture::__l2:: &) Line 1375 at c:\program files (x86)\microsoft visual studio 14.0\vc\include\type_traits(1375) [Inline Frame] qrenderdoc.exe!std::invoke(ReplayManager::OpenCapture::__l2:: &) Line 1443 at c:\program files (x86)\microsoft visual studio 14.0\vc\include\type_traits(1443) [Inline Frame] qrenderdoc.exe!std::_Invoke_ret(std::_Forced) Line 1461 at c:\program files (x86)\microsoft visual studio 14.0\vc\include\type_traits(1461) qrenderdoc.exe!std::_Func_impl<,std::allocator,void>::_Do_call() Line 214 at c:\program files (x86)\microsoft visual studio 14.0\vc\include\functional(214) [Inline Frame] qrenderdoc.exe!std::_Func_class::operator()() Line 279 at c:\program files (x86)\microsoft visual studio 14.0\vc\include\functional(279) qrenderdoc.exe!LambdaThread::process() Line 511 at c:\build\renderdoc\qrenderdoc\code\qrdutils.h(511) Qt5Core.dll!00007ffaba2a1d49() Qt5Core.dll!00007ffaba0d4c0f() Qt5Core.dll!00007ffaba0d5e6f() kernel32.dll!BaseThreadInitThunk() ntdll.dll!RtlUserThreadStart() ```

As far as I can tell from WrappedVulkan::Serialise_vkCreateRenderPass() the code simply does not handle VkRenderPassInputAttachmentAspectCreateInfo and ends up passing invalid parameters when creating separate passes for each individual subpass of the original. The structure chain of loadInfo contains all the same VkInputAttachmentAspectReference entries which were passed to the original vkCreateRenderPass call. In my particular case s (the subpass index) is 1, loadInfo.pSubpasses[0].inputAttachmentCount is 0, but the VkRenderPassInputAttachmentAspectCreateInfo's two pAspectReferences both reference the input attachments of subpass 0.

Ideally, RenderDoc should filter out the VkInputAttachmentAspectReference entries for each subpass and then zero out their subpass fields. It might also be OK to drop the structure from the chain entirely as its purpose is merely optimization.

Steps to reproduce

I can't share a capture of the application, but this is roughly how the render pass is created:

const auto attachments = std::vector<vk::AttachmentDescription>{
    vk::AttachmentDescription{{} /*flags*/, vk::Format::eD32SfloatS8Uint, /*...*/},
    /* other attachments */
};
const auto input_attachment_ref = vk::AttachmentReference{0, vk::ImageLayout::eShaderReadOnlyOptimal};
const auto subpasses = std::vector<vk::SubpassDescription>{
    vk::SubpassDescription{vk::SubpassDescriptionFlags{},
                           vk::PipelineBindPoint::eGraphics,
                           {1, &input_attachment_ref} /*input*/,
                           /*other attachments*/},
    vk::SubpassDescription{vk::SubpassDescriptionFlags{},
                           vk::PipelineBindPoint::eGraphics,
                           {} /*input*/,
                           /*other attachments*/},
};
const auto dependencies = std::vector<vk::SubpassDependency>{};
const auto input_aspect_refs = std::vector<vk::InputAttachmentAspectReference>{
     vk::InputAttachmentAspectReference{0 /*subpass*/, 0 /*inputAttachmentIndex*/, vk::ImageAspectFlagBits::eDepth}
};
const auto input_aspect_create_info = vk::RenderPassInputAttachmentAspectCreateInfo{input_aspect_refs};
const auto create_info = vk::RenderPassCreateInfo{vk::RenderPassCreateFlags{},
                                                  attachments,
                                                  subpasses,
                                                  dependencies,
                                                  &input_aspect_create_info /*pNext*/};

Environment

baldurk commented 2 years ago

Also closed by 9d18db6.