Checkerboard mode in Reblur causing issues with DLSS

[BattleAxeVR]

Hi, I'm having an issue with Reblur Checkerboard with DLSS and was wondering if it's a known issue or if there's a fix for it (something to do with viewport jittering scale factor?)

It makes sense that anyone who's using NRD also probably wants DLSS to upscale it, for performance and quality.

I've seen some ifdefs for WITH_DLSS in some of Nvidia's samples (it may be in RTXDI) so it's probably been noticed already, but I was hoping there might be a fix for this because checkerboard is simply too good for performance to ignore.

It just seems to mess up some parts of the image but not others, so my jitter values are properly set. In the geometry in the background, the oblique lines are properly anti-aliased and upscaled in both cases with no jaggies, but the statue of Venus herself here has too many visual anomalies to ignore (looks like old interlaced video seen on an LCD).

[BattleAxeVR]

Related sub-question, will DLSS be getting officially included in these samples at some point, now that DLSS itself is open to the public? I'd like to compare my implementation for any errors compared to a reference.

[dzhdanNV]

@BattleAxeVR Good news. A new big update is coming very-very soon. Since DLSS is publicly available now, it has been included (as a submodule) to the NRD sample. In other words, DLSS has been made a native part of the sample and I spent a lot of time making friendship between NRD/REBLUR and DLSS. Checkerboard with DLSS works well. Let's wait for this release, you will be able to check your implementation and, if needed, ask again.

[BattleAxeVR]

Thanks, I can't wait for the next code drop.

There's a comment in the code somewhere, something about "good denoisers do nothing with the environment map / skybox", I was wondering if that's just because of the potentially huge variance in picking out single pixels at random (or even importance-sampled like vk_pathtrace sample) in HDR images. So the approach with NRD here seems to be to sample an analytical sun using blue noise for direct lighting, and using (denoised) soft shadows in the lighting pass so you keep variance down. (apply the skybox only to the miss shader pixels and nowhere else). So the result looks quite different than the vk_pathtrace sample.

In my code, I managed to get decent denoised convergence by using a high mip LOD in the environment map, while skipping the soft shadow raycasts entirely (to save 1 ray cast / pixel and disable the shadow denoising). The reason I went this way is because I have dynamic video skies so I wanted the shadows in the scene to move around and not be so uniform as the sun, and give a result closer to what vk_pathtrace gives when you have an environment map. I believe RTXDI sample also uses HDR maps this way, but I'll have to check. Anyway this works well enough for now, sort of, with Reblur but Relax seems to have a really hard time with denoising the sky (even at high mip LODs).

No rush or anything to answer this, it's just a general question when/if you have time. I took a course in Monte Carlo a few years ago and what I remember from it is that when you pick your ray directions, importance sampling (cos hemisphere, or spec/diffuse BRDF sampling) helps you reduce variance, but of course that doesn't work in all cases (ex indoors with a window, if the brighter parts of the sky outside are occluded, then importance sampling the HDR skybox is a de-optimization, pointing your rays in directions it has no chance of hitting the brighter parts of the sky, instead of aiming at the window and taking whatever light it can comes from those directions

[BattleAxeVR]

Thanks for the update to 2.9 with DLSS.

It appears as though the sample is still a separate download, will that be released sometime soon? In a release .zip file I guess, like in 2.5?

[dzhdanNV]

Nope. Please, look at the 3rd (!) line of README file - it's the link to the sample. Actually, the main repo for you should become the NRD sample with all dependencies :)

[BattleAxeVR]

Thanks.

I got it now, but it looks like there's a bug. Enabling DLSS appears to have a broken camera input in Vulkan mode, tried various seetings like output res = 2.5k or 4K, different DLSS modes. D3D12 fails with an error about creating the pipeline too (I only care about Vulkan right now).

[dzhdanNV]

VK - seems to be a DLSS related issue D3D12 - upgrade to latest Win 10.

[dzhdanNV]

Yes, at least VK validation is not clean if DLSS is enabled. Callstacks point to DLSS. The DLSS team is aware and there is an internal bug. Feel free to move this issue into a separate "issue". I will let you know as soon as it gets resolved. Please, reformulate open questions regarding the original topic, if there are some... I will try to help and we will close the issue.

[dzhdanNV]

@BattleAxeVR Now it's unrelated to NRD. The NRD sample has been moved to a separate repo: https://github.com/NVIDIAGameWorks/NRDSample. Please, close this one and re-open there.