microsoft / DirectXShaderCompiler

This repo hosts the source for the DirectX Shader Compiler which is based on LLVM/Clang.
Other
3.04k stars 677 forks source link

Internal compiler error when attempting to link a non-library input #5736

Open EricLasotaRSE opened 1 year ago

EricLasotaRSE commented 1 year ago

Description DXC page faults if you attempt to feed a cs_6_3 input to the linker.

Steps to Reproduce Create this file and name it test.hlsl:

Texture2D<uint> texResource : register(t900);
RWTexture2D<uint> rwTexResource[] : register(u0, space2400);
[numthreads(8, 8, 1)]
void main(uint3 dtid : SV_DispatchThreadID, uint3 gtid : SV_GroupThreadID, uint3 gid : SV_GroupID, uint gindex : SV_GroupIndex)
{
    rwTexResource[0][dtid.xy] = texResource.Load(dtid.xyz);
}

Run these commands:

dxc.exe -T cs_6_3 -Fo test.bin test.hlsl
dxc.exe -link -T cs_6_3 -Fo test2.bin test.bin

Actual Behavior DXC will crash with this error: Internal compiler error: access violation. Attempted to read from address 0x0000000000000000

Environment

elasota commented 1 month ago

I did some investigation of this, but not sure of the best course of action.

It looks like what happens here is that a library target will output global variables for resources, but a compute shader will not. This means that if you feed a CS to the linker, then DxilLib::BuildGlobalUsage will add nothing to linkInfo->usedGVs, so DxilLinkJob::AddGlobals won't add it to the resource list.

Later, this results in GetResourcePropertyFromHandleCall (called from CollectShaderFlagsForModule) page faulting due to the CreateHandle op accessing an out-of-bounds resource index.

I am assuming that the linker expects that the input module doesn't use createHandle in the first place (and uses createHandleForLib instead), so I guess there are two options: Either allow createHandle in the inputs and fix up the resource indexes (allowing non-GV resource accesses - should these be allowed? Deduplicated?), or reject inputs that use createHandle.