Pretty simple shader fails on RDNA

[Vihior3]

Hi, I am part of a company that is developing a vulkan application and noticed that some specific shader code causes amdvlk64.dll to crash on RDNA but not other architectures. In RGA this just fails online mode and has to revert to offline. I already posted about this in the AMD community developer forum but it was suggested that this could be a better place for this issue.

The troublesome code involves sampling the same shader twice but in different places and using nonuniformEXT without changing the index it uses. This shader will trigger that behavior for example.

#version 450
#define WORK_ITEMS_X 16
#define WORK_ITEMS_Y 16
#define WORK_ITEMS_Z 1
layout(local_size_x = WORK_ITEMS_X, local_size_y = WORK_ITEMS_Y, local_size_z = WORK_ITEMS_Z) in;

// Header
#extension GL_ARB_separate_shader_objects : enable
#extension GL_ARB_shading_language_420pack : enable
#extension GL_EXT_nonuniform_qualifier : enable

// Attachments
layout(binding = 0, r16f) uniform image2D outColor;
layout(binding = 1) uniform sampler2D textureArrayRGBA[2048];

void main()
{
    uint index = gl_LocalInvocationIndex & 1;
    vec2 uv0 = vec2(gl_GlobalInvocationID.xy) / 512;
    vec2 uv1 = uv0 + 0.5;
    vec4 color0 = textureLod(textureArrayRGBA[nonuniformEXT(index)], uv0, 0.0);
    vec4 color1 = textureLod(textureArrayRGBA[nonuniformEXT(index)], uv1, 0.0);

    color0 += color1;
    imageStore(outColor, ivec2(gl_GlobalInvocationID.xy), color0);
}

RGA will create this output for this shader with gfx1030 and gfx906 selected, I just want to demonstrate that it works on older architectures:

----------------------------------------------------
Building Vulkan project "RDNA-fail-case" for gfx1030
----------------------------------------------------
rga.exe -s vulkan --isa "C:\Users\proce\OneDrive\Dokument\RadeonGPUAnalyzer\Projects\RDNA-fail-case\Output\Clone0\disassem.txt" --parse-isa --line-numbers --analysis "C:\Users\proce\OneDrive\Dokument\RadeonGPUAnalyzer\Projects\RDNA-fail-case\Output\Clone0\resourceUsage.csv" -b "C:\Users\proce\OneDrive\Dokument\RadeonGPUAnalyzer\Projects\RDNA-fail-case\Output\Clone0\codeobj.bin" --log "C:\Users\proce\AppData\Roaming\RadeonGPUAnalyzer\rga-cli-20210116-080532.log" --session-metadata "C:\Users\proce\OneDrive\Dokument\RadeonGPUAnalyzer\Projects\RDNA-fail-case\Output\Clone0\gfx1030_cliInvocation.xml" --asic gfx1030 --pso "C:/Users/proce/OneDrive/Dokument/RadeonGPUAnalyzer/Projects/RDNA-fail-case/Clone0/Pipeline0.cpso" --comp "C:\RGA\failcase.comp" 

Warning: failed to locate AMD's Vulkan driver on the system: amdvlk64.dll. Falling back to using the amdvlk binary that is packaged with RGA.
Target GPU detected:

gfx1030 (RDNA)
    AMD Radeon RX 6800
    AMD Radeon RX 6800 XT
    AMD Radeon RX 6900 XT
    AMD Radeon(TM) Graphics

Pre-compiling compute shader file (C:\RGA\failcase.comp) to SPIR-V binary (C:\Users\proce\AppData\Local\Temp\GPUOpen\rga\all-devices_rga-temp-out20444_comp.spv)... succeeded.
Building for gfx1030... 
Using Vulkan ICD from custom location: C:\RGA\utils\Vulkan\amdvlk/amdvlk64.dll

Warning: failed to locate AMD's Vulkan driver on the system: amdvlk64.dll. Falling back to using the amdvlk binary that is packaged with RGA.
Warning: falling back to building using Vulkan offline mode (-s vk-spv-offline). The generated ISA disassembly and HW resource usage information might be inaccurate. To get the most accurate results, adjust the pipeline state to match the shaders and rebuild.

Falling back to vk-spv-offline mode...
Target GPU detected:

gfx1030 (RDNA)
    AMD Radeon RX 6800
    AMD Radeon RX 6800 XT
    AMD Radeon RX 6900 XT
    AMD Radeon(TM) Graphics

Building for gfx1030... succeeded.
---------------------------------------------------
Building Vulkan project "RDNA-fail-case" for gfx906
---------------------------------------------------
rga.exe -s vulkan --isa "C:\Users\proce\OneDrive\Dokument\RadeonGPUAnalyzer\Projects\RDNA-fail-case\Output\Clone0\disassem.txt" --parse-isa --line-numbers --analysis "C:\Users\proce\OneDrive\Dokument\RadeonGPUAnalyzer\Projects\RDNA-fail-case\Output\Clone0\resourceUsage.csv" -b "C:\Users\proce\OneDrive\Dokument\RadeonGPUAnalyzer\Projects\RDNA-fail-case\Output\Clone0\codeobj.bin" --log "C:\Users\proce\AppData\Roaming\RadeonGPUAnalyzer\rga-cli-20210116-080532.log" --session-metadata "C:\Users\proce\OneDrive\Dokument\RadeonGPUAnalyzer\Projects\RDNA-fail-case\Output\Clone0\gfx906_cliInvocation.xml" --asic gfx906 --pso "C:/Users/proce/OneDrive/Dokument/RadeonGPUAnalyzer/Projects/RDNA-fail-case/Clone0/Pipeline0.cpso" --comp "C:\RGA\failcase.comp" 

Warning: failed to locate AMD's Vulkan driver on the system: amdvlk64.dll. Falling back to using the amdvlk binary that is packaged with RGA.
Target GPU detected:

gfx906 (Vega)
    AMD Radeon VII

Pre-compiling compute shader file (C:\RGA\failcase.comp) to SPIR-V binary (C:\Users\proce\AppData\Local\Temp\GPUOpen\rga\all-devices_rga-temp-out6900_comp.spv)... succeeded.
Building for gfx906... 
Using Vulkan ICD from custom location: C:\RGA\utils\Vulkan\amdvlk/amdvlk64.dll
succeeded.

I didn't use my systems driver here but I get the same behavior on the latest recommended driver (20.11.2) and the latest optional (20.12.1). However this used to work on drivers that are older than that and it also runs on nvidia cards.

Something else that is interesting is that having the VK_PIPELINE_CREATE_DISABLE_OPTIMIZATION_BIT flag in VkComputePipelineCreateInfo resolves the issue.

[AmitBM]

Hi Vihior3,

Thanks for reporting this - it seems like a possible driver/compiler issue.

Regarding the driver crash - Is there an executable that you can point us to which can be used to reproduce this behavior?

In addition, can you please share the RGA pipeline state file that you are using with your shader? You should be able to find it under the Clone0 subfolder of your project folder (generated under %DOCUMENTS%/RadeonGPUAnalyzer/Projects).

[Vihior3]

Yes, the pipeline state file can be found in this link: https://drive.google.com/file/d/1HtWWxV9p3OpiIEDbax37aKuOmYYGF_33/view?usp=sharing

And a small visual studio project that tries to create a pipeline (and crashes) can be found in this link: https://drive.google.com/file/d/15WyaznAkIJcsjopBQN6eOS_ob928ZmXt/view?usp=sharing

Note that the project will crash in amdvlk64.dll on RDNA (with the drivers I mentioned) otherwise it will print "Pipeline successfully created!!".

[AmitBM]

Thank you for the details - the issue will be investigated.

[AmitBM]

Hi Vihior3,

A driver fix has been pushed - it will be delivered in an upcoming Adrenalin Software release.

Thanks again for reporting this issue.

[Vihior3]

Glad to hear that and glad to help :) Looking forward to removing some workarounds for this issue.

[AmitBM]

Hi Vihior3,

The latest Adrenalin Software release should include a fix for this issue.

Can you please confirm that this is resolved for you?

[AmitBM]

This compute shader compiles successfully with the latest Adrenalin Software installed.