Reflections

[shelterx]

Not sure if this is a bug, you decide. But some surfaces are reflective/shiny with OpenGL/D3D, like some floors, for example like the one in the first room in the training level. I'm not sure if there should be similar "effects" when using EchelonRenderer?

[onnoj]

So, sadly this is where the scope of the renderer ends, and where hopefully future mods will pick up. In a raytracing renderer, reflections would need to be set on the material itself. This is where you'd need a proper RTX Remix mod that introduces new texturemaps to the scene.

So for a scene such as this one: the default material (which is just an albedo texture) won't reflect.

But, if the floor were to be replaced with a proper material (albedo + normal map + roughness map + metallic map), you'd get reflections: This is still the albedo texture, but with a manual tweak to it to have 0.01 roughness (ie almost a mirror) and slightly less metallic (not sure how to describe that, it's effectively organic versus shiny). With actual full PBR maps, you'd get a much more realistic look.

[onnoj]

I am however thinking about a way to at least some specific mirrors to work. There are some cases in the game (like the Hong Kong underground disco, and most bathroom mirrors) where mirrors are done by flagging a surface as 'PF_MIRROR' and not having a texture assigned. I think the renderer might be able to detect that and swap in a special 'mirror' texture.

[shelterx]

Ok, thanks for the response. However, I'm curious how the OpenGL renderer know that the floor is reflective? Hardcoded?

[onnoj]

Basically, in Deus Ex/UE, the renderer doesn't know anything about reflections. From its perspective, the reflection is just another room.

What happens in the game engine, is that when it's doing the occlusion pass (ie, it's figuring out which triangles are visible and which ones aren't), the algorithm the games used in the day made it very easy to have mirrors (and portals!). Normally, when traversing the BSP tree, if it'd hit a solid surface the engine just rasterizes it and sends it to the renderer as a bunch of triangles. But if that surface is marked a mirror, it basically travels back up the tree in reverse order with the Z-Axis flipped, sending a bunch more triangles to the renderer that make up the reflected space. ie, think of it as the game constructing an identical looking room behind the mirror surface, before sending it to the renderer.

In modern games, this trick doesn't work (which is why mirrors effectively disappeared from games until raytracing become more commonplace) because different algorithms are used; it just became much more efficient to sort your geometry front-to-back and let the gpu hardware figure it all out.

That same trick also doesn't work with raytracing anymore; for your GPU to be able to reliably shoot rays, the geometry has to be stable and sensible. Ie, with the same trick, the mirrored geometry would be sticking through actual geometry behind the mirror.

But, there are two things that we can do: A) For actual mirrors (which in Deus Ex tend to have a black or no texture and have their surface marked as 'mirror'), the Echelon renderer can try and intercept that, and then perform a texture replacement with a very reflective texture. B) For reflective surface textures, it's largely the same trick the engine does, but I'd have to look how Deus Ex/UE actually blends the texture with the reflection. This is probably where I can't do a texture swap, and where the original texture would have to be authored to be reflective.

[shelterx]

Thanks again, very nice explanation! So techically it's more difficult to do with RTX even if we know that a specific texture (or room) in a level should be slightly reflective.

[onnoj]

Yes, correct, something needs to tell the raytracer which parts of the texture is reflective. Right now, the raytracer only receives the albedo texture (ie, just contains the color).

And this is effectively what the RTX Toolkit is for, it allows you to replace an original texture with new ones that do contain that information: https://www.a23d.co/blog/different-maps-in-pbr-textures

ie, this (original albedo texture):

would need to be turned into this: (upscaled albedo) + (normal map) + (roughness)

As an example, those three were converted and upscaled by a neural network, and are not quite right. But, the real thing would look something like this: https://www.cgbookcase.com/textures/marble-08/

[shelterx]

Ok, that makes even more sense, You're not going to retexture the game, you provide the renderer which is more than enough. :)

[MamiyaOtaru]

roughness 0, flat normal map: