[REBLUR] Laggy reflections

[xardas110]

I am getting laggy reflections on my own project and on the NRD sample. To reproduce this on the NRD sample: FPS cap at 30, No normal map and metallic at 1, gives lots of lag near the reflection as show in the video below: https://user-images.githubusercontent.com/18483702/205492928-604dd7ac-17fb-4417-8477-76aa27ab5bf1.mp4

Is it supposed to be like this? How can one fix it in that case?

[dzhdanNV]

@xardas110 Thanks for the report! There was a bug. I have just submitted a fix. The root cause was in imprecisely computed normal-based reprojection confidence, which uses normals from the previous frame stored with reduced precision. As the result for flat surfaces the confidence was close to 1 (but not 1) what should not have happened, because the fallback to surface-based motion is inapplicable in this case.

Before NY I will release a big update, which is already WIP. I have added the new test based on your description to avoid such regressions in the future.

[xardas110]

Thanks for the fast response. Been experimenting with it in the past days so sorry for the late response. Your new update did fix the problem if the camera is near the wall with the reflection, but it also introduced the same problem when the camera is far away in my own project at least, hard to reproduce on the sample due to wall boundaries. I will do more testing and see if I can fix it somehow, just in case I have done something wrong. Other than that it seems like the reflections will always have some lag, if I am not mistaken?

[dzhdanNV]

Please, show me a video. Please, also update to v4.0.0 when you have time. Several reprojection issues have been fixed in the latest version.

[xardas110]

So I have updated to the latest version(v4.0.1) and the only problem now with the reflections is lag on curved surfaces. Flat surfaces project the reflections without any lag now, but both in my own project and the NRD sample I have laggy reflections on curved surfaces, is this normal?

NRD sample(FPS cap 30), lag on curved reflective surface: https://user-images.githubusercontent.com/18483702/210129002-4b83de65-3704-466f-aa72-0b4dff308df2.mp4

My sample, lag only happens when it's close to the reflective sphere: https://user-images.githubusercontent.com/18483702/210129130-3cbc3f8a-0620-4137-9c93-5a8f703e490d.mp4

My sample, flat surface, no lag: https://user-images.githubusercontent.com/18483702/210129695-713449cd-0464-45ce-ad6e-c09ab7bacc64.mp4

[dzhdanNV]

NRD sample(FPS cap 30), lag on curved reflective surface...

Thanks for digging and checking the latest. Yes, it doesn't look correct, I will investigate.

[dzhdanNV]

Observations:

• lag in NRD sample in your video most likely related to wrong signs in thin lens equation in case of concave mirrors
• lag in your sample can be related to imprecision of passed normals. Try with high quality normal encoding = 4, please.

Nevertheless I found several bugs and fixed them:

• temporal stabilization pass now gets curvature in FP16, not a repacked one in 8-bit (important when the camera is very close to a large smooth object)
• fixed race conditions while finding min hitT for specular tracking purposes (can be a reason of your problems too)
• fixed signs of object and image distances in thin lens equation in case of concave mirrors (negative curvature)
• improved curvature estimation in case of high parallax
• dropped some hacks

I need to do more testing and then I will update GitHub (this week according to my plan).

[xardas110]

The video is running with normal encoding = 3, this was changed inside the cmakelists.txt inside the denoiser root folder, just mentioning it in case that's the incorrect way of doing it. I've tried 4 as well with no luck. The normal value also look best at encoding = 2, 3 and 4 gives less visible normals.

[dzhdanNV]

It doesn't make a lot of sense, because 3 & 4 assumes 16-bit per normal component. OK, let's wait when I update GitHub (in a few days) to see the latest in action.

[dzhdanNV]

@xardas110 Please, check v4.0.2.

[xardas110]

Same problem

[dzhdanNV]

• can the problem be reproduced in NRD sample?
• if yes, does it happen on concave or convex mirrors?

[dzhdanNV]

if yes, does it happen on concave or convex mirrors?

Can you add a test and send it to me, please? To achieve this:

• unfold TESTS section
• reproduce the problem
• press add button
• press on appeared test at the end of the list and ensure that visuals match your expectations
• send me one of .bin files from Tests folder depending on the scene, you worked with

[xardas110]

Sorry for my vague reply earlier. I tested on my own project and it had the same problem, but with less lag. When testing on the NRD sample the concave mirror problem is gone, but I can still see ghosting on convex mirrors. Other than convex mirrors the denoiser seem to work flawlessly.

I added a few tests here for convex mirrors: ShaderBalls.zip

[dzhdanNV]

Thanks. I see the problem. OK, will do my best to fix it.

[Osintedx]

I've somewhat found a Fix to this.

[dzhdanNV]

It's worth trying the latest upcoming version (this week). But I know that REBLUR / RELAX don't reproject ideally on silhouettes of curved objects, when the camera is close to the object. I will keep this one open, because I'm very interested in proper fixing of this problem (by improving curvature math and reprojection, not by accelerating accumulation).

[dzhdanNV]

I've somewhat found a Fix to this.

Closing then ;) Hopefully in the future I will be able to improve curvature in general.