DXIL: Deadlock when loading `RayDesc` from `ByteAddressBuffer`

[MarijnS95]

Our previous shader compiler came from https://github.com/microsoft/DirectXShaderCompiler/commit/0392e60dbc87bf8aa5cbfe826622b517fe39fde0 which worked fine. We now downloaded new blobs from https://github.com/microsoft/DirectXShaderCompiler/commit/ea3623fdf712fdab2d247e9f0c4e41daef43e9b8 / https://ci.appveyor.com/project/dnovillo/directxshadercompiler/builds/47177077 and observe a deadlock with the following pattern:

repro.hlsl:

[numthreads(32, 8, 1)] void main(uint2 threadId
                                 : SV_DispatchThreadID) {
    ByteAddressBuffer buffer = ResourceDescriptorHeap[NonUniformResourceIndex(10)];
    RayDesc result = buffer.Load<RayDesc>(sizeof(RayDesc) * 1);
}

Run dxc from the above (Windows or Linux) download, and observe that the command runs indefinitely:

./dxc-artifacts/bin/dxc -E main -T cs_6_6 repro.hlsl -Fo test

---

I understand RayDesc to be a simple POD, and copying that into a local RayDesc2 struct and loading that doesn't deadlock.

[llvm-beanz]

sigh

My least favorite pass strikes again:

cannot flatten hlsl intrinsic.Assertion failed: (false && "cannot flatten hlsl intrinsic."), function RewriteCall, file ScalarReplAggregatesHLSL.cpp, line 2761.

[MarijnS95]

Oh indeed, on a newer compiler (from https://github.com/microsoft/DirectXShaderCompiler/commit/6287d513d1f791715a9e3b1ad01e8287aeca0d6b) this is now an assert.

[MarijnS95]

It's still an assert for the simple example above, but usually a deadlock on the bigger, massively-parallel-compiled shaders in our project.

Fortunately some unrelated issue we were facing (multiple identical compile invocations returning vastly different DXIL blobs) has magically been solved, and we can now upgrade to the latest DXC :)

[damyanp]

Looks like this still repros: https://godbolt.org/z/z11rb6E6E

[llvm-beanz]

We're clearly giving the builtin RayDesc struct some extra handling, but you can workaround this by writing your own RayDesc struct:

https://godbolt.org/z/M8ov38c9c