[RELAX] Specular reflections behaving incorrectly upon camera translation

[LeonBrands]

We recently updated from NRD 2.9.0 to 3.7.0, and started noticing issues with specular reflections that seem to interact with surrounding shadows incorrectly upon camera translation.

See the video below: https://youtu.be/PuE9Us28NZU&vq=hd1080

Left view = NRD on Right view = NRD off

As is visible in the video (although a little difficult to see through compression), the shadow of the box gets dragged along camera translation around the area of the floor that is specularly lit. We didn't have this problem in previous versions of NRD, but we also didn't notice the issue in the 3.7.0 sample, which seems to imply that this is a problem on our side. I've tried comparing our NRD integration to the official sample but unfortunately to no avail. I'm posting here in the hope that someone with more experience with the API might be able to identify the issue.

In case it is helpful, I've included a file with our integration, thank you in advance for any help 🙂 yar_nrd.txt

[dzhdanNV]

Hi @LeonBrands ! I believe your NRD integration is correct. I managed to reproduce your problem in the sample. You can easily do it too:

• run Shader Balls
• set metalness = 1
• set roughness = 0.1
• uncheck Normal map
• fly out (kinda zoom out) a bit
• lock FPS to 30
• start strafing and observe "dark" ghosting

I see in your video that in your case roughness is bigger, but I hope that these issues are related...

What's next?

• I have fixed (ok, significantly improved) behavior in the sample (in the case described above)
• I would like to know linear roughness value in your case, please
• Could you try to repro in Bistro Interior, please? I believe you can start with tests 32 and 44
• I will release v3.8 by the end of this week (with many fixes + specific fix for you if the root cause is different and I will be able to find it 🙂)

[LeonBrands]

Thank you for your swift response! I'll run the tests you mentioned and get back to you ASAP

[LeonBrands]

The linear roughness on the floor and boxes in my video is 0.55294, I tried the tests and while a bit more nuanced, I do notice a similar type of ghosting yes!

[dzhdanNV]

@LeonBrands Please, try v3.8.0. It will be our new baseline. This issue is not a regression. I have been trying to tune the algorithm to behave "good on average" under any conditions. And, unfortunately, in your case we have hit a situation, where an older version behaves better in this case. If the issue persists, please, try to repro in the NRD sample, add a test and drop BistroInterior.bin file here.

[dzhdanNV]

Please, try the latest...

[LeonBrands]

Sorry I missed your messages! I'll give it a try tomorrow morning

[LeonBrands]

The ghosting is unfortunately still present; https://youtu.be/4f-jxYdWTlo

I did some testing and got the same effect with the NRDSample (test 44) using Bistro Interior, FPS capped to 30, and ray samples to 8: https://youtu.be/PpUwOpmn6l0. It appears that more noise (lowering the sample count) seems to somehow result in less ghosting.

[dzhdanNV]

Thanks for reproducing the problem in your environment and the NRD sample. I will try to improve. Please, give me a few days. I will return back to you soon.

[dzhdanNV]

@LeonBrands How many rays per pixel do you use in your app?

[LeonBrands]

The engine uses an implementation of RTXDI; approximately 2 rays (initial sampling + final visibility), but with the checkerboard configuration, so ultimately 1 ray/pixel. The signal from RTXDI is fairly clean.

[LeonBrands]

We do use heavy upscaling (DLSS + our own VR upscaling tech), so I believe RTXDI and NRD only run on approximately a 900x900 resolution

[dzhdanNV]

Can you, please, do a simple experiment for me (assuming that you use REBLUR)? Find this code (it's in PrePass):

            if( ( GetLuma( s ) > minHitDistLuma && w != 0.0 )
                minHitDist = min( minHitDist, ExtractHitDist( s ) );

And replace to:

            if( w != 0.0 )
                minHitDist = min( minHitDist, ExtractHitDist( s ) );

Is there an improvement?

[LeonBrands]

Oh sorry I forgot to mention that, no we use RELAX

[dzhdanNV]

I see. Is it difficult to switch to REBLUR for comparison? In any case I will redirect the issue to RELAX dev. In test 44 (with described above conditions) smearing in RELAX is higher.

[LeonBrands]

Hmm we don't currently have REBLUR implemented but I'll try to find some time to do so this week...

[dzhdanNV]

Thanks. No rush.

[dzhdanNV]

In case of many RPP passing min hitT across all paths fixes the issue. It's hard to say how relevant this fact is to your case, but straightforward averaging works badly, because "far" samples can outweigh sample with smaller hit distances. It breaks virtual motion tracking. It worked better before, because in older versions motion was shifted more towards "surface" motion and you was lucky enough to avoid bumping into issues related to it :)

[LeonBrands]

Hmm in our case our hitT is simply computed as the distance from the world position to the light that was sampled, so there's really only one hitT that we could be passing along

[dzhdanNV]

Please, try to do this simple experiment in your environment - find this line in RELAX/PrePass:

        specularIllumination.a = max(1.0e-6, lerp(specularHitT, minHitT, NoV));

And replace with:

        specularIllumination.a = max(1.0e-6, minHitT);

If you want to compare BEFORE/AFTER in real-time then use this:

        specularIllumination.a = max(1.0e-6, lerp(specularHitT, minHitT, gDebug > 0.0 ? 1.0 : NoV));

You will need to pass something into CommonSettings::debug.

[LeonBrands]

Tried that; seems to have fairly little effect unfortunately (left debug = 1.0, right debug = 0.0) https://youtu.be/PwJeMhHImKw

[dzhdanNV]

Thanks! You use default settings specularPrepassBlurRadius = 50.0f, right? In the NRD sample with 8 rpp and/or locked FPS in test 44 it fixes reprojection behavior.

(not claiming anything, just gathering info)

[LeonBrands]

Correct, I set all my settings to their defaults to make comparisons easier

[dzhdanNV]

Please, update to the latest. I believe this issue is fixed now.

[LeonBrands]

Thank you, I will try that

[dzhdanNV]

@LeonBrands Hi! Any news?

[dzhdanNV]

Closing. RELAX has evolved a lot. Feel free to reopen if the issue is still there.