Meshes whose textures were marked as "Sky Textures" can't be captured nor replaced

[Ekozmaster]

Tested in U2 and Sonic Adventures DX. If one mark the sky texture as such, captures won't contain the meshes and even if you handwrite a replacement for that mesh_HASH, the replacement won't take place. Tested with the latest dxvk build #235, but has been happening for a couple months.

Steps for the missing mesh in the capture:

• Launch any compatible game
• Find the sky texture and categorize it as such
• Capture frame in USD
• Open it in OV, the mesh won't be there.

Steps for testing the replacement not working:

• Launch any compatible game
• Don't mark as sky texture yet
• Capture the mesh, setup a replacement for it. I've made a sphere mesh with a HDR map all setup: Models.zip
• If it works in game, now categorize the texture as sky texture and the replacement won't work anymore.

Bonus:

• If you slap a sun light in the mesh, it won't cross the dome even explicitly setting the mesh to single sided and to not cast shadows (See the attached mesh).

Expected behaviour

• Objects whose textures have been marked as "Sky Texture" could be captured and replaced as any other.
• Replaced sky meshes would still count as "miss rays" by the raytracing shader, thus, being more physically precise and also saving performance.
• The sun would light up the scene as usual, even if its not a custom directional light, just the game's original light translated to the runtime.

[nv-ajaus]

Hi @Ekozmaster , can you please file a bug for the capture issue and a separate bug for the replacement issue in the https://github.com/NVIDIAGameWorks/rtx-remix project? An an example with a specific application and logs could be helpful.

[Ekozmaster]

Hi, @nv-ajaus , sure, though, I won't be at home until monday 13th. The steps mentioned + attached zip didn't reproduce the issue?

For the replacement issue:

You can hop on our NFSU2 repo and set it up on the game as in the README, then proceed to just mark the current sky texture as sky and the issue happens. The game swaps sky texture every ~5 minutes. It currently has a replacement for the sky mesh and a hdri texture (the ones in the attached zip), but once one categorize them as sky texture, the replacements are ignored and the default original mesh + texture is rendered.

This is really bad for us because, not being able to set those expensive replacements as sky means the whole world is enclosed in an emissive sphere which bounces light all around and significantly impacts performance, rather than considering that as "miss rays" for the raytracing shader as far as I understand.

As for the capture issue:

Really any game with a sky being rendered has this issue, the generated capture.usd has no sky defined if it was categorized as such. Tried with U2, Sonic & SuperTuxKart 🐧 . By cloning our U2 repo, this should also happen. Just have to mark the sky as sky texture and make a capture of the scene. The sky won't be present in OV

[nv-ajaus]

Hi @Ekozmaster , since it's been a while, I'm going to close this issue for now. If you want to continue working on these issues, and you have the time, can you please file a github issue for the replacement problem and a github issue for the capture problem on the https://github.com/NVIDIAGameWorks/rtx-remix/issues page?