icculus / mojoshader

Use Direct3D shaders with other 3D rendering APIs.
https://icculus.org/mojoshader/
zlib License
138 stars 36 forks source link

Indexed (non unrolled) loops in vertex shaders break when the array size is over 4 elements #69

Open Kleadron opened 3 weeks ago

Kleadron commented 3 weeks ago

I'm working on a tile based terrain system in XNA/FNA and I've implemented directional and point lights for it. I have my light parameters in my effect file defined as such:

#define MAXLIGHTS 8
int NumLights = 0;
float4 LightPosType[MAXLIGHTS];
float4 LightColorRadii[MAXLIGHTS];

LightPosType uses the xyz components for position or direction, and the w component to determine if a light is a directional or point light. LightColorRadii defines the color in the rgb components and the radius in the a component.

and my loop to iterate these properties looks like this:

    for(int i = 0; i < NumLights; i++)
    {
        float4 lightPosType = LightPosType[i];
        float4 lightColRadii = LightColorRadii[i];
        CalcLambertian(worldPosition, worldNormal, lightPosType, lightColRadii, lightColor);
    }

The expected result of this code, viewed from XNA/DX9. image

In FNA however, one of two things happens depending on the order I declared my light parameters arrays. If LightPosType is declared before LightColorRadii, all lights after the first will be at the 0 0 0 position. (The first light in the scene is the directional lighting.) image

if I declare LightColorRadii first, and LightPosType second, then all lights after the first one are placed correctly but have bogus colors and radius. image

These issues go away if I add the [unroll] attribute before the loop, but ideally I shouldn't need to do this, and it will take up more shader instructions. They also go away if the array size was set to 4, but this restricts the number of lights I can render in a single pass. The behavior does not change if I am compiling the shader for vs_2_0 or vs_3_0. The behavior also does not change if I go over 8 lights. 16 lights will work on XNA but will still break on FNA.

The results were shown using FNA3D's DX11 backend. The same result also shows up with FNA3D's OpenGL backend.

What's even more exciting is when I compile the effect with optimization enabled, it will crash my GPU driver and turn my desktop black for a few moments, but only in FNA!

I believe this is a mojoshader issue as the parameters are being set correctly and this issue does not happen in DX9.

Kleadron commented 3 weeks ago

Here is the source file, disassembly, and compiled FXB files if they are of any help. This was compiled with the following command using the 2010 DirectX SDK: fxc main.fx /T fx_2_0 /Od /Fo main.fxb /Fx main.txt main.zip

kg commented 3 weeks ago

fwiw I've had success with large arrays in shaders before in FNA3D mojoshader, so there might be something subtle going on here. IIRC Kleadron also tried with the 2018 version of fxc on my suggestion, and it was still broken. I looked at the shader disassembly for the loop and it seems correct (albeit weird) to me.