NVIDIAGameWorks / rtx-remix

Combined repo for the RTX-Remix runtime
https://www.nvidia.com/en-us/geforce/rtx-remix/
MIT License
1.32k stars 68 forks source link

[Runtime Bug]: colorTexture2 Color & Alpha Not Blending with colorTexture1 - Star Wars Empire at War (2006) #512

Open Arrokoth7323 opened 1 month ago

Arrokoth7323 commented 1 month ago

Describe the bug

Context:

For the game Star Wars Empire at War (Directx9.0c), we have a odd issue. There is multiple shaders which use fixed function calls to blend textures to produce one output.

  1. Space Fog of War: A fog of war mask texture is generated by the CPU to indicate in real time using a alpha channel, where units can see on a 2d plane. This is seen as a corresponding white texture with an alpha of where units can see is generated. This is combined with a generic diffuse texture for a grid which should have its color and alpha "blended" with the fog of war texture.

  2. Land Terrain Textures: A mask texture is generated by the CPU to indicate on a cell of the map where a certain terrain texture should be painted. This is seen as a corresponding white texture. This is combined with a generic diffuse texture for the type of terrain (grass/rock/dirt/etc.) which should have its color "blended" with the mask texture. The game splits a "cell" of a terrain into chunks based on where a certain terrain texture is painted. Hence a separate grid like mesh to created, split from the original square cell. The texture is then painted onto the new mesh within the now split cell.

Issue Observed:

When handling two color textures. Remix will read colorTexture1 and will display said colorTexture1 without blending the color or alpha of colorTexture2.

colorTexture1 & colorTexture2 will be read by Remix and present in the list of textures on the side for Space Fog of War. However for Land Terrain Texture only colorTexture1 will be present. If you ignore the skydome texture, then mark the terrain white mask texture as Skybox, the proper blended Terrain Texture becomes visible. However it is not read by Remix at said point and hence no render calls (lighting) are made on it, so this "workaround" is pointless.

I ultimately believe these two issues are very similar in scope and hence could share a single fix.

Code:

As mentioned prior, Empire at Wars shaders are all purpose built with fallback fixed function techniques to ensure compatibility with less-powerful graphics cards of the time. See below for each of the above issues, the corresponding shader technique code, to get an idea of the fixed function calls used.

  1. Space Fog of War (Shader Name: SpaceFogOfWar.fx):

    
    technique t1
    <
    string LOD="FIXEDFUNCTION";
    >
    {
    pass t1_p0
    {
        SB_START
    
            ZWriteEnable=false;
            ZFunc=lessequal;    
            AlphaBlendEnable=true;
            DestBlend = INVSRCALPHA;
            SrcBlend = SRCALPHA;
    
            CullMode=none;
    
            // FF Vertex pipeline
            Lighting=false;
    
            // FF Pixel pipeline
            AddressU[0] = CLAMP;
            AddressV[0] = CLAMP;
    
            AddressU[1] = WRAP;
            AddressV[1] = WRAP;
    
            TexCoordIndex[1] = CAMERASPACEPOSITION;
            TextureTransformFlags[1] = COUNT2;
            ColorOp[0]=SELECTARG1;
            ColorArg1[0]=TEXTURE;
            AlphaOp[0]=SELECTARG1;
            AlphaArg1[0]=TEXTURE;
    
            ColorOp[1]=SELECTARG1;
            ColorArg1[1]=CURRENT;
            ColorArg2[1]=TEXTURE;
    
            AlphaOp[1]=MODULATE;
            AlphaArg1[1]=TEXTURE;
            AlphaArg2[1]=CURRENT;
    
            ColorOp[2]=Disable;
            AlphaOp[2]=Disable;
    
        SB_END        
    
        VertexShader = NULL;
        PixelShader = NULL;
    
        Texture[0] = (m_FOWTexture);
        Texture[1] = (GridTexture);
        TextureTransform[1] = 
        (
            mul(
                m_viewInv,
                float4x4(
                    float4(200*m_FOWTexU.x,200*m_FOWTexV.x,0,0),
                    float4(200*m_FOWTexU.y,200*m_FOWTexV.y,0,0),
                    float4(0,0,1,0),
                    float4(0,0,0,1))
                )
        );
    }
    
    // cleanup pass
    pass t1_cleanup < bool AlamoCleanup = true; >
    {
        SB_START
    
            AddressU[0] = WRAP;
            AddressV[0] = WRAP;
    
            TexCoordIndex[0] = 0;
            TextureTransformFlags[0] = DISABLE;
            TexCoordIndex[1] = 1;
            TextureTransformFlags[1] = DISABLE;
    
        SB_END        
    }

}

2. Land Terrain Textures (**Shader Name:** _TerrainRenderBump.fx_): 

technique t3 < string LOD="FIXEDFUNCTION";

{ pass t3_p0 { SB_START

        // General Render States
        ZEnable=true;
        ZWriteEnable=true;
        ZFunc=lessequal;

        // Blend Texture
        MinFilter[0]=LINEAR;
        MagFilter[0]=LINEAR;
        MipFilter[0]=LINEAR;

        TexCoordIndex[0] = CAMERASPACEPOSITION;
        TextureTransformFlags[0] = COUNT2;

        ColorOp[0]=SELECTARG1;
        ColorArg1[0]=TEXTURE;
        AlphaOp[0]=SELECTARG1;
        AlphaArg1[0]=TEXTURE;

        // Diffuse+Gloss Texture
        MinFilter[0]=LINEAR;
        MagFilter[0]=LINEAR;
        MipFilter[0]=LINEAR;

        TexCoordIndex[1] = CAMERASPACEPOSITION;
        TextureTransformFlags[1] = COUNT2;

        ColorOp[1]=MODULATE2X;
        ColorArg1[1]=DIFFUSE;
        ColorArg2[1]=TEXTURE;
        AlphaOp[1]=SELECTARG1;
        AlphaArg1[1]=CURRENT;

        ColorOp[2]=Disable;
        AlphaOp[2]=Disable;

        /* 
        Attempt at gloss-masked specular...
        ColorOp[2]=MODULATEALPHA_ADDCOLOR;
        ColorArg1[2]=CURRENT;
        ColorArg2[2]=SPECULAR;
        AlphaOp[2]=SELECTARG1;
        AlphaArg1[2]=CURRENT;
        */

    SB_END        

    VertexShader = NULL;
    PixelShader = NULL;

    Texture[0] = (blendTexture);
    TextureTransform[0] = 
    (
        mul(
            m_viewInv,
            float4x4(
                float4(blendTexScale.x,0,0,0),
                float4(0,blendTexScale.y,0,0),
                float4(0,0,1,0),
                float4(blendTexOffset.x,blendTexOffset.y,0,1))
            )
    );

    Texture[1] = (diffuseTexture);
    TextureTransform[1] = 
    (
        mul(
            m_viewInv,
            float4x4(   float4(diffuseTexU.x,diffuseTexV.x,0,0),
                        float4(diffuseTexU.y,diffuseTexV.y,0,0),
                        float4(diffuseTexU.z,diffuseTexV.z,1,0),
                        float4(diffuseTexU.w,diffuseTexV.w,0,1))
            )
    );
    // Material colors
    MaterialAmbient = (materialDiffuse);
    MaterialDiffuse = (materialDiffuse);
    MaterialEmissive = (float4(0,0,0,0));
    MaterialSpecular = (materialSpecular);
    MaterialPower = (16.0f);
}

// cleanup pass
pass t3_cleanup < bool AlamoCleanup = true; >
{
    SB_START

        TexCoordIndex[0] = 0;
        TexCoordIndex[1] = 1;
        TextureTransformFlags[0] = DISABLE;
        TextureTransformFlags[1] = DISABLE;

    SB_END        
}

}


### **Helpful Info:**

From a conversation with NV_Mark on the _RTX Remix Showcase_ Discord, he mentioned within the opaque material, specifically:
chooseTextureArgument(textureArg1, surface.textureColorArg1Source, albedo, surfaceInteraction.vertexColor.rgb, tFactor.rgb, f16vec3(1.0));
chooseTextureArgument(textureArg2, surface.textureColorArg2Source, albedo, surfaceInteraction.vertexColor.rgb, tFactor.rgb, f16vec3(1.0));
chooseTextureOperationColor(albedo, surface.textureColorOperation, textureArg1, textureArg2);

There is an example of colorTexture1 & colorTexture2 blending. Perhaps however it is not being handed right/is bugged in ```chooseTextureOperationColor``` of ```src\dxvk\shaders\rtx\concept\surface_material\opaque_surface_material_blending.slangh```. 

The ray portal material for Portal was also mentioned as an example of colorTexture2 usage.

### **Image Breakdown:**

For further context for the above two issue cases, see the below images from the DirectX SDK (June 2010) debugger alongside images from in-game with RTX Remix enabled.

1. Space Fog of War:
**Fog of War Mask Texture (colorTexture1):**
![Fog of War Mask Texture](https://github.com/NVIDIAGameWorks/rtx-remix/assets/168481954/8e66ff2c-4a24-4036-83ca-6bf23eebb793)
**Grid Texture (colorTexture2):**
![Grid Texture](https://github.com/NVIDIAGameWorks/rtx-remix/assets/168481954/542a3883-2320-40de-8102-64806f34ec3f)
**Fog of War Mask Texture combined with Grid Texture:**
![hldZQ2rh93](https://github.com/NVIDIAGameWorks/rtx-remix/assets/168481954/aa40c4ff-eac1-420e-80bd-374e69f5cf2b)
**Remix's interpretation of the Shader. Failing to combine the two textures. Only showing Fog of War Mask Texture (colorTexture1)**
![20240521144532_1](https://github.com/NVIDIAGameWorks/rtx-remix/assets/168481954/d559f3b6-9360-47c0-9132-0e74f3bc9241)

2. Land Terrain Textures:
**Land Terrain Mask Texture (colorTexture1):**
![l8LS0CZJfo](https://github.com/NVIDIAGameWorks/rtx-remix/assets/168481954/19a7bf3f-d7d5-4a5f-8edd-a62bcd3da932)
**Terrain Diffuse Texture (colorTexture2):**
![rEPap2D5em](https://github.com/NVIDIAGameWorks/rtx-remix/assets/168481954/5427e4b1-eb17-48bd-95d6-6e2358e7c466)
**Land Terrain Mask Texture _BEFORE_ combination with Terrain Diffuse Texture:**
![uPlN14SUT3](https://github.com/NVIDIAGameWorks/rtx-remix/assets/168481954/85af2cf4-9576-4412-8842-47ad40e3d80f)
**Land Terrain Mask Texture _AFTER_ combination with Terrain Diffuse Texture:**
![AsPowerBar_iBUXVdD5hg](https://github.com/NVIDIAGameWorks/rtx-remix/assets/168481954/c45914f7-2b6b-478d-b738-885b71076798)
**Remix's interpretation of the Shader. Failing to combine the two textures. Only showing Land Terrain Mask Texture (colorTexture1)**
![20240521162754_1](https://github.com/NVIDIAGameWorks/rtx-remix/assets/168481954/0dc5f12f-4590-460b-9641-d302c7890ab9)

### **Useful Files:**

[StarWarsG_d3d9.log](https://github.com/NVIDIAGameWorks/rtx-remix/files/15396073/StarWarsG_d3d9.log)
[rtx.conf.txt](https://github.com/NVIDIAGameWorks/rtx-remix/files/15396082/rtx.conf.txt)
[dxvk.conf.txt](https://github.com/NVIDIAGameWorks/rtx-remix/files/15396084/dxvk.conf.txt)

### How do you reproduce the bug?

1. Launch Star Wars Empire at War
2. Enter into any Space Skirmish Map
3. Observe if Fog of War is blended with Grid Texture color + alpha. 

- If solid gray plane observed then issue reproduced.
- If grid like plane observed then issue not reproduced, working as expected.

1. Launch Star Wars Empire at War
2. Enter into any Land Skirmish Map
3. Observe if Terrain texture mask is blended with Terrain diffuse map.

- If solid white plane observed then issue reproduced
- If terrain texture (such as grass or dirt) is visible, then issue not reproduced, working as expected.

### What is the expected behavior?

For Space the runtime should properly blend the mask texture whos alpha indicates visible space with the grid texture which indicates not visible space.

For Land the runtime should properly blend the mask texture with the terrain texture. The mask texture color indicates where a terrain texture should be present.

### Version

0.5.1
MarkEHenderson commented 1 month ago

Note: I suspect these draw calls would work if they were categorized as terrain, but since the colorTexture1 is unstable (dynamically generated on the cpu), there's no way to reliably categorize them.

I suspect we'll need to enable categorization and replacement by colorTexture2 to really make this effect work.

Alternatively, @Arrokoth7323 would it be possible to get the game to swap the textures, so that colorTexture1 is the stable texture (either the grid or the grass texture) and colorTexture2 is the mask? If so, you could probably label them as terrain to get stuff to work properly.

Arrokoth7323 commented 1 month ago

Note: I suspect these draw calls would work if they were categorized as terrain, but since the colorTexture1 is unstable (dynamically generated on the cpu), there's no way to reliably categorize them.

I suspect we'll need to enable categorization and replacement by colorTexture2 to really make this effect work.

Alternatively, @Arrokoth7323 would it be possible to get the game to swap the textures, so that colorTexture1 is the stable texture (either the grid or the grass texture) and colorTexture2 is the mask? If so, you could probably label them as terrain to get stuff to work properly.

Marking as Terrain Before Swapping colorTexture 1 & colorTexture2 Positions in Shader

Plan of Action I took here was to mark the Land Terrain Mask and Fog of War Mask texture as Terrain, then reboot. Runtime Terrain Baking was on, screenshots are after the reboot. 20240522102738_1

  1. Land Terrain Textures: With Land Terrain there is much more success than with Space. It never dawned on me what I was seeing was actually the proper texture... but with one slight issue. When you mark the Land Terrain Mask texture as Terrrain, the proper Land Terrain diffuse map (grass/rock/dirt/etc.) becomes visible. However the texture is very bright and moves with the camera as displayed below. After Marking Terrain Mask Texture (colorTexture1) as Terrain: 20240522102720_1 ![Streamable Video Link of this first process in action] (For Reference) This is the displayed Terrain Diffuse map texture: rEPap2D5em

  2. Space Fog of War: After Marking Fog of War Mask Texture (colorTexture1) as Terrain: 20240522102824_1 (Note: I have tried marking the Space Fog of War Mask and Space Fog of War Grid textures as both Terrain. This does not change the above result, even after reboot) After Un-Marking Fog of War Mask Texture (colorTexture1) as Terrain: 20240522102839_1

Something to note, whenever you load a new map in Space, the Fog of War Mask texture generates a new hash, aka becomes a new texture. This is because when starting a new map, your units are in a different position compared to other maps, hence the generated Mask texture for what you see is different. Simply going to each map and marking the initial Fog of War Mask texture is not a stable solution either. This is because when starting a new map your units will not always be in the same position as you last started that exact same map (blame small craft like fighters always spawning in a random position within a region). But if they happen to be, then yes the Mask texture is recognized from before and automatically displayed as marked Terrain.

Another thing to note, when marking either the Terrain Mask or Space Fog of War Mask textures as Skybox, the original correct version of the blended texture becomes visible. They are promptly ignored by the renderer however, as in light does not cast onto them anymore (because its Skybox now).

20240522122217_1 Image below is for Terrain Mask marked as Skybox. You will see the proper green color, not over illuminated like in prior pictures (probably in-part due to no light casting on it). 20240522122119_1

Marking as Terrain After Swapping colorTexture 1 & colorTexture2 Positions in Shader

So for Space and Terrain, when swapping colorTexture2 to become the new colorTexture1, said Space Grid or Terrain Diffuse texture is visible initially, but still not blended with their corresponding mask alphas. This means Space has the entire plane used for Fog of War as now a Grid Texture with no cutouts where units currently see. For Terrain this means the ground shows the correct terrain textures (albeit not terribly stable as seen in below video) but without the proper blending from the Mask, hence all textures look "blocky", not smooth like painted with a brush. There is no change in outcome when marking either the Space Grid or Terrain Diffuse and the corresponding mask texture as terrain and rebooting.

  1. Land Terrain Textures: Before Marking Terrain Diffuse Texture (colorTexture2 now as colorTexture1) as Terrain: 20240522124845_1 After Marking Terrain Diffuse Texture (colorTexture2 now as colorTexture1) as Terrain: 20240522124858_1 ![Streamable Video Link of this second process in action]

  2. Space Fog of War: Before Marking Fog of War Grid Texture (colorTexture2 now as colorTexture1) as Terrain: 20240522130919_1 After Marking Fog of War Grid Texture (colorTexture2 now as colorTexture1) as Terrain: 20240522131026_1

Updated colorTexture2 now as colorTexture1 Shader Code

What I've done for each of the below is essentially maintain the original operations on each texture, but simply change their order of operation. So colorTexture1 is now colorTexture2 vice versa, but the operations taking place on said textures is still the same behavior if you compare to my original post's Shader code.

  1. Land Terrain Textures (Shader Name: TerrainRenderBump.fx):

    technique t1
    <
    string LOD="FIXEDFUNCTION";
    >
    {
    pass t1_p0
    {
        SB_START
    
            // General Render States
            ZEnable=true;
            ZWriteEnable=true;
            ZFunc=lessequal;
    
            AddressU[0] = CLAMP;
            AddressV[0] = CLAMP;
    
            TexCoordIndex[0] = CAMERASPACEPOSITION;
            TextureTransformFlags[0] = COUNT2;
    
            ColorOp[0]=MODULATE2X;
            ColorArg1[0]=DIFFUSE;
            ColorArg2[0]=TEXTURE;
    
        SB_END        
    
        VertexShader = NULL;
        PixelShader = NULL;
    
        Texture[0] = (diffuseTexture);
        TextureTransform[0] = 
        (
            mul(
                m_viewInv,
                float4x4(   float4(diffuseTexU.x,diffuseTexV.x,0,0),
                            float4(diffuseTexU.y,diffuseTexV.y,0,0),
                            float4(diffuseTexU.z,diffuseTexV.z,1,0),
                            float4(diffuseTexU.w,diffuseTexV.w,0,1))
                )
        );
    
        // Material colors
        MaterialAmbient = (float4(0,0,0,0)); //materialDiffuse);
        MaterialDiffuse = (float4(1,1,1,1)); //materialDiffuse);
        MaterialEmissive = (float4(0,0,0,0));
        MaterialSpecular = (float4(0,0,0,0)); //materialSpecular);
        MaterialPower = (16.0f);
    
    }
    
    // cleanup pass
    pass t3_cleanup < bool AlamoCleanup = true; >
    {
        SB_START
    
            TexCoordIndex[0] = 0;
            TexCoordIndex[1] = 1;
            TextureTransformFlags[0] = DISABLE;
            TextureTransformFlags[1] = DISABLE;
    
        SB_END        
    }
    }
  2. Space Fog of War (Shader Name: SpaceFogOfWar.fx):

    
    technique t1
    <
    string LOD="FIXEDFUNCTION";
    >
    {
    pass t1_p0
    {
        SB_START
    
            ZWriteEnable=false;
            ZFunc=lessequal;    
            AlphaBlendEnable=true;
            DestBlend = INVSRCALPHA;
            SrcBlend = SRCALPHA;
    
            CullMode=none;
    
            // FF Vertex pipeline
            Lighting=false;
    
            // FF Pixel pipeline
            AddressU[0] = WRAP;
            AddressV[0] = WRAP;
    
            AddressU[1] = CLAMP;
            AddressV[1] = CLAMP;
    
            TexCoordIndex[0] = CAMERASPACEPOSITION;
            TextureTransformFlags[0] = COUNT2;
    
            ColorOp[0]=SELECTARG1;
            ColorArg1[0]=CURRENT;
            ColorArg2[0]=TEXTURE;
    
            AlphaOp[0]=MODULATE;
            AlphaArg1[0]=TEXTURE;
            AlphaArg2[0]=CURRENT;
    
            ColorOp[1]=SELECTARG1;
            ColorArg1[1]=TEXTURE;
            AlphaOp[1]=SELECTARG1;
            AlphaArg1[1]=TEXTURE;
    
            ColorOp[2]=Disable;
            AlphaOp[2]=Disable;
    
        SB_END        
    
        VertexShader = NULL;
        PixelShader = NULL;
    
        Texture[0] = (GridTexture);
        Texture[1] = (m_FOWTexture);
        TextureTransform[0] = 
        (
            mul(
                m_viewInv,
                float4x4(
                    float4(200*m_FOWTexU.x,200*m_FOWTexV.x,0,0),
                    float4(200*m_FOWTexU.y,200*m_FOWTexV.y,0,0),
                    float4(0,0,1,0),
                    float4(0,0,0,1))
                )
        );
    }
    
    // cleanup pass
    pass t1_cleanup < bool AlamoCleanup = true; >
    {
        SB_START
    
            AddressU[1] = WRAP;
            AddressV[1] = WRAP;
    
            TexCoordIndex[0] = 1;
            TexCoordIndex[1] = 0;
            TextureTransformFlags[0] = DISABLE;
            TextureTransformFlags[1] = DISABLE;
    
        SB_END        
    }

}

Arrokoth7323 commented 1 month ago

In response to a comment by NV_Mark on the RTX Remix Showcase Discord, see below results of testing these modified shaders in-game without Remix enabled (Note: Im not Shader expert, consider these to be crude edits):

If you use your new shaders without remix, do they work correctly? just wanted to verify

Testing Modified Shaders In-Game Without Remix Enabled:

They do not function properly in-game with Remix disabled. As of now I have tried two methods for reversing the order of colorTexture1 & 2 in the shaders:

  1. The way I described in the above comment, where I do not change the operations being taken on the associated textures, simply change the order. To where colorTexture2 becomes colorTexture1 so it now executes first vice versa.

With this method colorTexture2 is visible in-game blended with colorTexture1. So for Land Terrain this means the gray mask texture is visible. With Remix to reproduce this using the updated shader, you initially see the Land Terrain diffuse texture, once you mark it as Terrain then it becomes the gray mask.

20240522154121_1 20240522154157_1

  1. The second way which I just tried now is simply swapping which textures outright like below:

Original:

Texture[0] = (m_FOWTexture);
Texture[1] = (GridTexture);

Swapped:

Texture[0] = (GridTexture);
Texture[1] = (m_FOWTexture);

This proves not much better, as now you see the Mask Texture using a mix of the Alpha/Color of the Grid/Terrain Diffuse texture.

20240522154405_1 20240522154344_1

Overall neither of my shaders edits have produced results. But one thing is clear, no matter the colorTexture1, whatever is set as colorTexture2 ends up being what is rendered in-game as the final visible texture. Similar to how marking a texture as Terrain with Remix makes the colorTexture2 visible on Land Terrain (not in Space, in Space its makes the entire plane used for the texture become dark grey.

MarkEHenderson commented 1 month ago

However the texture is very bright and moves with the camera as displayed below. I don't expect the terrain baker to continue working properly once the initial texture you marked is changed by the game - as then the terrain isn't getting any new draw calls, which can cause lots of problems. I'm surprised it turns out so bright though.

I don't understand enough about how the .fx files are turned into actual shading and rendering code. Most likely, you'll need to swap some stuff higher up, as well as associated bits like the texture transform. If you can find any guides or write ups for how this file type is supposed to be used, you may find this easier to swap correctly. If you can get the shaders working outside of Remix with the texture orders swapped, then I would expect it to become much easier to correctly handle them within remix (whether with terrain or some other option)

Arrokoth7323 commented 1 month ago

However the texture is very bright and moves with the camera as displayed below. I don't expect the terrain baker to continue working properly once the initial texture you marked is changed by the game - as then the terrain isn't getting any new draw calls, which can cause lots of problems. I'm surprised it turns out so bright though.

I don't understand enough about how the .fx files are turned into actual shading and rendering code. Most likely, you'll need to swap some stuff higher up, as well as associated bits like the texture transform. If you can find any guides or write ups for how this file type is supposed to be used, you may find this easier to swap correctly. If you can get the shaders working outside of Remix with the texture orders swapped, then I would expect it to become much easier to correctly handle them within remix (whether with terrain or some other option)

However the texture is very bright and moves with the camera as displayed below. I don't expect the terrain baker to continue working properly once the initial texture you marked is changed by the game - as then the terrain isn't getting any new draw calls, which can cause lots of problems. I'm surprised it turns out so bright though.

I don't understand enough about how the .fx files are turned into actual shading and rendering code. Most likely, you'll need to swap some stuff higher up, as well as associated bits like the texture transform. If you can find any guides or write ups for how this file type is supposed to be used, you may find this easier to swap correctly. If you can get the shaders working outside of Remix with the texture orders swapped, then I would expect it to become much easier to correctly handle them within remix (whether with terrain or some other option)

So a bit of an update so far. After working with user Gaukler, an expert with shaders from this title, he produced a shader modification for SpaceFogOfWar.fx which he felt was the proper way to reverse the colorTexture2 with 1 order whilst preserving the intended affect. See below results of using this updated shader with Remix disabled and enabled. I will post results for the modifications to the TerrainRenderBump.fx shader come next week.

Remix Disabled with Updated Shader Code to Properly Reverse colorTexture2 with 1 (SpaceFogOfWar.fx):

20240523170745_1

You can see above the Shader produces the correctly blended grid texture whilst calling the grid texture first within the shader. The original code has the mask texture being called first. See the updated shader code below.

Remix Enabled with Updated Shader Code to Properly Reverse colorTexture2 with 1 (SpaceFogOfWar.fx):

20240523170612_1

The above image is before marking the now initially visible grid texture as terrain. Before you would just see the gray mask texture with the proper alpha indicating where units see. Now you see a grid texture without the mask texture's alpha indicating where units see. Hence the grid texture now covers the entire plane used for Fog of War rendering.

20240523170017_1

The above image you can see both the grid texture and generated mask texture are marked as terrain.

However, this re-orientation of textures to ensure the grid is generated first has inadvertently fixed our issue in a different way. This plane used to render the Fog of War is more or less meant to be UI, hence it should not have light cast onto it. The way to disable light casting onto it is to mark the grid texture (which is not generated by CPU and will always be the same hash no matter the map or units present) as Sky Texture. This will produce the properly blended grid texture as shown in the pictures below:

20240523172523_1 20240523172527_1

Marking as WorldSpaceUI or UI texture will not produce the same results, it will simply hide the texture outright. I am not sure if this is ultimately how the correct manner of approaching this issue. But the fact we don't have to re-tag a mask texture every time we enter a new map is awesome and hence I'm willing to put aside the issue with SpaceFogOfWar.

Will post another reply after testing next week with the updated TerrainRenderBump shader occurs. Thank you for your help so far!!

Updated Shader Code (SpaceFogOfWar.fx):

technique t1
<
    string LOD="FIXEDFUNCTION";
>
{
    pass t1_p0
    {
        SB_START

            ZWriteEnable=false;
            ZFunc=lessequal;    
            AlphaBlendEnable=true;
            DestBlend = INVSRCALPHA;
            SrcBlend = SRCALPHA;

            CullMode=none;

            // FF Vertex pipeline
            Lighting=false;

            // FF Pixel pipeline    
            AddressU[0] = WRAP;
            AddressV[0] = WRAP;

            AddressU[1] = CLAMP;
            AddressV[1] = CLAMP;

            TexCoordIndex[0] = CAMERASPACEPOSITION;
            TextureTransformFlags[0] = COUNT2;
            ColorOp[0]=SELECTARG1;
            ColorArg1[0]=TEXTURE;
            AlphaOp[0]=SELECTARG1;
            AlphaArg1[0]=TEXTURE;

            TexCoordIndex[1] = 0;    
            ColorOp[1]=SELECTARG2;
            ColorArg1[1]=CURRENT;
            ColorArg2[1]=TEXTURE;

            AlphaOp[1]=MODULATE;
            AlphaArg1[1]=TEXTURE;
            AlphaArg2[1]=CURRENT;

            ColorOp[2]=Disable;
            AlphaOp[2]=Disable;

        SB_END        

        VertexShader = NULL;
        PixelShader = NULL;

        Texture[0] = (GridTexture);
        Texture[1] = (m_FOWTexture);
        TextureTransform[0] = 
        (
            mul(
                m_viewInv,
                float4x4(
                    float4(200*m_FOWTexU.x,200*m_FOWTexV.x,0,0),
                    float4(200*m_FOWTexU.y,200*m_FOWTexV.y,0,0),
                    float4(0,0,1,0),
                    float4(0,0,0,1))
                )
        );
    }

    // cleanup pass
    pass t1_cleanup < bool AlamoCleanup = true; >
    {
        SB_START

            AddressU[0] = WRAP;
            AddressV[0] = WRAP;

            TexCoordIndex[0] = 0;
            TextureTransformFlags[0] = DISABLE;
            TexCoordIndex[1] = 1;
            TextureTransformFlags[1] = DISABLE;

        SB_END        
    }

}
Arrokoth7323 commented 1 month ago

However the texture is very bright and moves with the camera as displayed below. I don't expect the terrain baker to continue working properly once the initial texture you marked is changed by the game - as then the terrain isn't getting any new draw calls, which can cause lots of problems. I'm surprised it turns out so bright though.

I don't understand enough about how the .fx files are turned into actual shading and rendering code. Most likely, you'll need to swap some stuff higher up, as well as associated bits like the texture transform. If you can find any guides or write ups for how this file type is supposed to be used, you may find this easier to swap correctly. If you can get the shaders working outside of Remix with the texture orders swapped, then I would expect it to become much easier to correctly handle them within remix (whether with terrain or some other option)

Remix Disabled with Updated Shader Code to Properly Reverse colorTexture2 with 1 (TerrainRenderBump.fx):

20240528102731_1

You can see above the Shader produces the correctly blended terrain diffuse texture whilst calling the terrain diffuse texture first within the shader. The original code has the mask/blend texture being called first. See the updated shader code below.

Remix Enabled with Updated Shader Code to Properly Reverse colorTexture2 with 1 (TerrainRenderBump.fx):

20240528193234_1

The above image is with the new shader code AND with each terrain diffuse texture's alpha channel removed. This produces an awesome result where terrain is visible on all maps. However there is still two separate draw calls occurring, evident by the fact for each painted terrain texture, Remix detects the diffuse AND the mask as seperate textures, similar to the SpaceFogOfWar issue. It remains an issue that the two are not being properly blended, which creates a very blocky appearance to the land.

We still also have an issue where the meshes the terrain is painted onto flickers when moving the camera. I believe this is tied to the fact there really isnt a proper "blend" going on at the edges of the different textures now. I believe this because on maps where there is simply 1 terrain texture, no flickering ever occurs on moving the camera.

20240528132526_1

Picture above is a map with a single terrain texture, no flickering occurs when moving the camera.

![Streamable Video Link]

Video above is a demonstration of the diffuse textures showing but no blending and a persistent flickering issue when moving the camera. As well as what happens when you mark the diffuse as Terrain (very bright color and moves with camera).

Updated Shader Code (TerrainRenderBump.fx):

technique t3
<
    string LOD="FIXEDFUNCTION";
>
{
    pass t3_p0
    {
        SB_START

            // General Render States
            ZEnable=true;
            ZWriteEnable=true;
            ZFunc=lessequal;

            // Blend Texture
            MinFilter[0]=LINEAR;
            MagFilter[0]=LINEAR;
            MipFilter[0]=LINEAR;

            TexCoordIndex[0] = CAMERASPACEPOSITION;
            TextureTransformFlags[0] = COUNT2;

            ColorOp[0]=SELECTARG1;
            ColorArg1[0]=TEXTURE;
            AlphaOp[0]=SELECTARG1;
            AlphaArg1[0]=TEXTURE;

            // Diffuse+Gloss Texture
            MinFilter[0]=LINEAR;
            MagFilter[0]=LINEAR;
            MipFilter[0]=LINEAR;

            TexCoordIndex[1] = CAMERASPACEPOSITION;
            TextureTransformFlags[1] = COUNT2;

            ColorOp[1]=MODULATE2X;
            ColorArg1[1]=DIFFUSE;
            ColorArg2[1]=CURRENT;
            AlphaOp[1]=SELECTARG1;
            AlphaArg1[1]=TEXTURE;

            ColorOp[2]=Disable;
            AlphaOp[2]=Disable;

            /* 
            Attempt at gloss-masked specular...
            ColorOp[2]=MODULATEALPHA_ADDCOLOR;
            ColorArg1[2]=CURRENT;
            ColorArg2[2]=SPECULAR;
            AlphaOp[2]=SELECTARG1;
            AlphaArg1[2]=CURRENT;
            */

        SB_END        

        VertexShader = NULL;
        PixelShader = NULL;

        Texture[1] = (blendTexture);
        TextureTransform[1] = 
        (
            mul(
                m_viewInv,
                float4x4(
                    float4(blendTexScale.x,0,0,0),
                    float4(0,blendTexScale.y,0,0),
                    float4(0,0,1,0),
                    float4(blendTexOffset.x,blendTexOffset.y,0,1))
                )
        );

        Texture[0] = (diffuseTexture);
        TextureTransform[0] = 
        (
            mul(
                m_viewInv,
                float4x4(   float4(diffuseTexU.x,diffuseTexV.x,0,0),
                            float4(diffuseTexU.y,diffuseTexV.y,0,0),
                            float4(diffuseTexU.z,diffuseTexV.z,1,0),
                            float4(diffuseTexU.w,diffuseTexV.w,0,1))
                )
        );
        // Material colors
        MaterialAmbient = (materialDiffuse);
        MaterialDiffuse = (materialDiffuse);
        MaterialEmissive = (float4(0,0,0,0));
        MaterialSpecular = (materialSpecular);
        MaterialPower = (16.0f);
    }

    // cleanup pass
    pass t3_cleanup < bool AlamoCleanup = true; >
    {
        SB_START

            TexCoordIndex[0] = 0;
            TexCoordIndex[1] = 1;
            TextureTransformFlags[0] = DISABLE;
            TextureTransformFlags[1] = DISABLE;

        SB_END        
    }
}
NV-LL commented 1 month ago

REMIX-3117 for tracking