Crash when running my app

GabeRundlett commented 6 months ago

In latest origin/development, I get the following crash when I try to launch my app:

Crash detected, current frames:
        [00007FFE5203D000] DebugBreak
                C:\Windows\System32\KERNELBASE.dll
        [00007FFCFAB84420] Detail::Break
                C:\dev\downloads\GPU-Reshape\Bin\MSVC\Debug\GRS.Backends.Vulkan.Layer.dll
                C:\dev\downloads\GPU-Reshape\Source\Libraries\Common\Source\Assert.cpp line 54
        [00007FFCFA907930] PipelineCompiler::CompileCompute
                C:\dev\downloads\GPU-Reshape\Bin\MSVC\Debug\GRS.Backends.Vulkan.Layer.dll
                C:\dev\downloads\GPU-Reshape\Source\Backends\Vulkan\Layer\Source\Compiler\PipelineCompiler.cpp line 267
        [00007FFCFA907EA0] PipelineCompiler::WorkerCompute
                C:\dev\downloads\GPU-Reshape\Source\Backends\Vulkan\Layer\Source\Compiler\PipelineCompiler.cpp line 142
        [00007FFCFA90A7E0] `Detail::DelegateCreator<void (__cdecl PipelineCompiler::*)(void *)>::MakeFrameProxy<{PipelineCompiler::WorkerCompute,0,0}>'::`2'::<lambda_1>::operator()
                C:\dev\downloads\GPU-Reshape\Source\Libraries\Common\Include\Common\Delegate.h line 81
        [00007FFCFA908420] `Detail::DelegateCreator<void (__cdecl PipelineCompiler::*)(void *)>::MakeFrameProxy<{PipelineCompiler::WorkerCompute,0,0}>'::`2'::<lambda_1>::<lambda_invoker_cdecl>
                C:\dev\downloads\GPU-Reshape\Source\Libraries\Common\Include\Common\Delegate.h line 81
        [00007FFCFAAE3120] Delegate<void __cdecl(void *)>::Invoke
                C:\dev\downloads\GPU-Reshape\Source\Libraries\Common\Include\Common\Delegate.h line 48
        [00007FFCFAAE36A0] DispatcherWorker::ThreadEntry
                C:\dev\downloads\GPU-Reshape\Source\Libraries\Common\Include\Common\Dispatcher\DispatcherWorker.h line 68
        [00007FFCFAADFE90] std::invoke<void (__cdecl DispatcherWorker::*)(void),DispatcherWorker *>
                C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\include\type_traits line 1756
        [00007FFCFAADF600] std::thread::_Invoke<std::tuple<void (__cdecl DispatcherWorker::*)(void),DispatcherWorker *>,0,1>
                C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\include\thread line 61        [00007FFD0AC12EE0] register_onexit_function
        [00007FFE53CA2560] BaseThreadInitThunk
        [00007FFE54ACAA20] RtlUserThreadStart

it appears that it fails to instrument a compute shader. I'm not sure why this would happen. it also appears that only compute/graphics are supported. I have some RT shaders, and maybe this is related. Will RT support come, or could they maybe just be ignored?

miguel-petersen commented 6 months ago

Hi Gabe,

Thanks for the report, I see the assert. Though in this case I think it should be downgraded to only the PipelineMissingShaderKey. Curious as to why it's getting in there.

Regarding RT shaders, they aren't supported just yet, but it's definitely coming. Mesh shader support is landing quite soon, RT shaders are next. The challenge with them is how to handle local root signatures.

Reshape "should" let RT shaders passthrough without issue. One thing to keep in mind is that if you're instrumenting for, say, initialization, and an RT shader initializes a resource, Reshape will not catch it just yet.

miguel-petersen commented 3 months ago

Hi @GabeRundlett.

The development branch has seen a ton of changes go in, including a fix for your issue. Sorry it took so long!

It does require .NET SDK 8 now, with VS22. There'll be a release soon.

GPUOpen-Tools / GPU-Reshape

Crash when running my app #62