baldurk / renderdoc

RenderDoc is a stand-alone graphics debugging tool.
https://renderdoc.org
MIT License
9.02k stars 1.35k forks source link

Crash when a non-multiview `VkRenderPass` is created with `VkRenderPassMultiviewCreateInfo::subpassCount == 0` #2747

Closed glebov-andrey closed 2 years ago

glebov-andrey commented 2 years ago

Description

The RenderDoc application crashes while loading a capture containing a non-multiview VkRenderPass which was created with a VkRenderPassMultiviewCreateInfo structure in its VkRenderPassCreateInfo structure chain.

The stack trace: (click to expand) ``` nvoglv64.dll!00007ffa57138f09() nvoglv64.dll!00007ffa57138e88() nvoglv64.dll!00007ffa57137ffd() renderdoc.dll!WrappedVulkan::Serialise_vkCreateRenderPass(ReadSerialiser & ser, VkDevice_T *) Line 1119 at c:\build\renderdoc\renderdoc\driver\vulkan\wrappers\vk_misc_funcs.cpp(1119) renderdoc.dll!WrappedVulkan::ProcessChunk(ReadSerialiser & ser, VulkanChunk chunk) Line 3261 at c:\build\renderdoc\renderdoc\driver\vulkan\vk_core.cpp(3261) renderdoc.dll!WrappedVulkan::ReadLogInitialisation(RDCFile * rdc, bool storeStructuredBuffers) Line 2639 at c:\build\renderdoc\renderdoc\driver\vulkan\vk_core.cpp(2639) renderdoc.dll!VulkanReplay::ReadLogInitialisation(RDCFile * rdc, bool storeStructuredBuffers) Line 225 at c:\build\renderdoc\renderdoc\driver\vulkan\vk_replay.cpp(225) renderdoc.dll!ReplayController::PostCreateInit(IReplayDriver * device, RDCFile * rdc) Line 2148 at c:\build\renderdoc\renderdoc\replay\replay_controller.cpp(2148) renderdoc.dll!ReplayController::CreateDevice(RDCFile * rdc, const ReplayOptions & opts) Line 2115 at c:\build\renderdoc\renderdoc\replay\replay_controller.cpp(2115) renderdoc.dll!CaptureFile::OpenCapture(const ReplayOptions & opts, std::function progress) Line 371 at c:\build\renderdoc\renderdoc\replay\capture_file.cpp(371) qrenderdoc.exe!ReplayManager::run(int proxyRenderer, const QString & capturefile, const ReplayOptions & opts, std::function progress) Line 452 at c:\build\renderdoc\qrenderdoc\code\replaymanager.cpp(452) [Inline Frame] qrenderdoc.exe!ReplayManager::OpenCapture::__l2::::operator()() Line 58 at c:\build\renderdoc\qrenderdoc\code\replaymanager.cpp(58) [Inline Frame] qrenderdoc.exe!std::_Invoker_functor::_Call(ReplayManager::OpenCapture::__l2:: &) Line 1375 at c:\program files (x86)\microsoft visual studio 14.0\vc\include\type_traits(1375) [Inline Frame] qrenderdoc.exe!std::invoke(ReplayManager::OpenCapture::__l2:: &) Line 1443 at c:\program files (x86)\microsoft visual studio 14.0\vc\include\type_traits(1443) [Inline Frame] qrenderdoc.exe!std::_Invoke_ret(std::_Forced) Line 1461 at c:\program files (x86)\microsoft visual studio 14.0\vc\include\type_traits(1461) qrenderdoc.exe!std::_Func_impl<,std::allocator,void>::_Do_call() Line 214 at c:\program files (x86)\microsoft visual studio 14.0\vc\include\functional(214) [Inline Frame] qrenderdoc.exe!std::_Func_class::operator()() Line 279 at c:\program files (x86)\microsoft visual studio 14.0\vc\include\functional(279) qrenderdoc.exe!LambdaThread::process() Line 511 at c:\build\renderdoc\qrenderdoc\code\qrdutils.h(511) Qt5Core.dll!00007ffabbb81d49() Qt5Core.dll!00007ffabb9b4c0f() Qt5Core.dll!00007ffabb9b5e6f() kernel32.dll!BaseThreadInitThunk() ntdll.dll!RtlUserThreadStart() ```

As far as I can tell from inspecting the loadInfo structure which gets passes to CreateRenderPass() by WrappedVulkan::Serialise_vkCreateRenderPass(), the function MakeSubpassLoadRP() incorrectly patches the VkRenderPassMultiviewCreateInfo structure here: https://github.com/baldurk/renderdoc/blob/048b85ffc840b01fc398f032626868f4596d3d86/renderdoc/driver/vulkan/wrappers/vk_misc_funcs.cpp#L71-L72 It assumes that the existence of this structure in the chain indicates that multiview.subpassCount is equal to info.subpassCount, but according to the spec, that's not the case. The documentation for VkRenderPassMultiviewCreateInfo states that

  • subpassCount is zero or the number of subpasses in the render pass.
  • pViewMasks is a pointer to an array of subpassCount view masks, where each mask is a bitfield of view indices describing which views rendering is broadcast to in each subpass, when multiview is enabled. If subpassCount is zero, each view mask is treated as zero.

Also, the documentation for VkRenderPassCreateInfo does not disallow the existence of VkRenderPassMultiviewCreateInfo for non-multiview passes. The Vulkan Validation Layers don't complain about this either.

Steps to reproduce

Sadly I can't share a capture of the application, but the render pass is created roughly like this:

std::vector<vk::AttachmentDescription> attachments = /*...*/;
std::vector<vk::SubpassDescription> subpasses = /*...*/;
std::vector<vk::SubpassDependency> dependencies = /*...*/;
std::vector<std::uint32_t> view_masks{};
std::vector<std::uint32_t> correlation_masks{};
if (is_multiview /* false in this case */) {
    /* fill view_masks and correlation_masks */
}
const auto multiview_create_info = vk::RenderPassMultiviewCreateInfo{
    view_masks,
    {} /*viewOffsets*/,
    correlation_masks};
const auto create_info = vk::RenderPassCreateInfo{vk::RenderPassCreateFlags{},
                                                  attachments,
                                                  subpasses,
                                                  dependencies,
                                                  &multiview_create_info};
const auto render_pass = device.createRenderPassUnique(create_info, nullptr /*allocator*/);

Simply filling view_masks with subpasses.size() zeroes fixes the crash.

Environment

baldurk commented 2 years ago

Also closed by 9d18db6.