Refraction denoising

[nyorain]

Would it be possible to use NRD to denoise refractions? Just using the specular reblur denoiser seems promising, but for low-roughness surfaces it is overly noisy under camera motion. It looks like the temporal accumulation has some false rejects, probably due to the specular parallax logic in there that does not work well for refraction anymore. This is visible even when there is no actual refraction happening and it is basically just transmission. My test case is a planar water-like surface.

Do you have any pointers what would need to be changed in the denoiser to make this possible? (I realize that the NRD sample and other pathtracing implementations have alternative mechanisms like SHaRC in place for this usecase but I feel like it should be solvable via the denoiser just like specular reflections).

[dzhdanNV]

I would start with a short answer - a reflections denoiser can't be used for denoising refractions in all cases, at least because their motion is different (the most problematic part is changing of the ray direction). IMO, to avoid denoising refractions (or at least to greatly simplify the task) better do something like:

• trace and denoise opaque
• trace refractions reusing already denoised data if a hit is inside the screen (>95% of cases)
• optionally use any form of radiance caching, like SHARC

But it could work in some cases, RTX REMIX, for example, uses NRD for refraction denoising but the disocclusion threshold is greatly relaxed to ~0.1 (instead of 0.01) on surfaces with refractions using IN_DISOCCLUSION_THRESHOLD_MIX input.

I could also suggest to experiment with PSR in this case.

I don't know all the details, so it's hard to provide more explicit guidance. Do you merge reflections and refractions into one input? Or do you denoise refractions separately? Could you attach a video of the issue (better with NRD validation enabled, if available), please?

[nyorain]

Thanks for your quick answer! I did already denoise refractions separately from reflections. Sadly, I'm not allowed to share a video.

I ended up implementing the approach you outline. It has problems with rough refractions (which always require denoising). In that case PSR will not work either. However, using the specular reblur variant on that input data works reasonably well, at the cost of an additional denoising instance (we now need opaque diffuse+spec, transparent spec for reflection and separate transparent spec for refraction) and some potential overblurring when the surface is not rough. To fix that, I lerp between the non-denoised and denoised variant now in the end when applying refraction, depending on primary (transparent) surface roughness.

I'm still wondering how much effort a reblur variant for refraction would be and believe it would improve results, potentially allowing us to get rid of the extra denoising instance and offering a more generic solution. But the current solution works, so I'll close the issue.

[dzhdanNV]

Refraction denoising requires research. Going spatial-only simplifies everything, but if accumulation is needed it becomes complicated. Refractions can move in a weird way, motion estimation can be expensive. I'm not promising anything, but I will keep in mind that there is a desire to get a refraction denoiser.