Oidn doesn't work with Bidirectional path tracing renderings

[Dade916]

It looks like Oidn doesn't work with Bidirectional path tracing renderings (with Metropolis sampler and spectral rendering) like this one:

or this one:

The output buffer it is just the same of the input, apparently untouched, even with a simple rendering like Cornell University box.

I'm not sure if this is a known limitation.

[atafra]

We actually haven't tested OIDN with such a renderer yet. I think bidirectional path tracing shouldn't cause any issues, but Metropolis and spectral probably do. Could you please provide the exact input buffers, e.g. as EXR files? The Cornell box would be fine as a start.

[Dade916]

I have attached a zip with the result of 64 samples per pixel rendering of Cornell Box with BiDir+Metropolis sampler+Blackman Harris pixel filter. There are the normal and denoised .PNG/.EXR files plus albedo and normal .EXR files.

Oidn seems to not touch the image at all (?)

bidir-files.zip

[atafra]

Thanks, I can reproduce the issue. The noise pattern looks different from non-MLT sampling (as expected), and since the network was not trained with MLT, it does not consider this as noise. Could you re-render this image with BiDir but without Metropolis?

[Dade916]

Sure, I will post the renderings with Random and Sobol sampler tomorrow. I will try also with enabled/disabled pixel filtering.

[atafra]

That will be perfect, thanks!

[Ultimatemau]

Sure, I will post the renderings with Random and Sobol sampler tomorrow. I will try also with enabled/disabled pixel filtering.

From our experience with testing the denoiser pixel filters have to be turned off. Similar reasons apply to Metropolis type algorithms. The main reason is the correlation between pixels due to e.g. pixel filtering, path space correlation due to metropolis sampling. In Hyperion they opt out of pixel filtering and implemented Filtered Importance Sampling + Adaptive sampling (to get a nice equal noise distribution across the entire image for better denoising). https://www.yiningkarlli.com/projects/hyperiondesign/hyperiondesign.pdf -> Section 5.x

[atafra]

Yes, correlation between pixels is likely causing the issue. But the denoiser does seem to work well with at least some pixel filters (e.g. Blackman-Harris + Sobol sampler + unidirectional path tracer in LuxCoreRender, Gaussian in Blender Cycles).

[Dade916]

I attached several tests but the problem seems to be there in all cases. The file names should be somewhat self-esplicative. The most basic is bidir-random-nofilter-noadpt-32spp.tar.gz:

• Bidirectional path tracing
• Random sampler
• No pixel filter
• No adaptive sampling
• 32 or 64 samples per pixel (2x tar.gz)

I have the filling, the source of the problem is in the noise pattern coming from the light paths. We have also a light tracer render engine in LuxCore (aka forward ray tracer) for debugging. I can post a render with that too if you want.

bidir-random-blackman.tar.gz bidir-random-nofilter.tar.gz bidir-random-nofilter-noadpt.tar.gz bidir-random-nofilter-noadpt-32spp.tar.gz bidir-random-nofilter-noadpt-64spp.tar.gz bidir-sobol-blackman.tar.gz

[atafra]

Thanks, this is very useful! Could you please render it with the unidirectional path tracer too, for reference?

[atafra]

Hmm, I just had a quick look into the LuxCore code, and I'm now a bit confused. Is this actually traditional bidirectional path tracing or VCM?

[Dade916]

We have both but my actual tests are about normal BiDir (VCM is just experimental and not "officially" supported). I guess some confusion may arise by the fact that BiDirVCM render engine is a derived class of normal BiDir.

I will post the path tracer test tomorrow.

It could be interesting to do a test also with Maxwell or Indigo Render. They are the only other 2 widely available BiDir renders I can think to.

[atafra]

OK, thanks for the clarification! I would like to compare the noise patterns of the uni- and bidirectional path tracers. I'm still wondering why denoising doesn't work with BiDir if there's no correlation between the neighboring pixels.

[atafra]

Mitsuba also has a BiDir renderer, so I tried it out and... OIDN can actually denoise it without any issues!

Input ("bdpt" integrator, box filter, 64 spp, latest Mitsuba version from the git repo):

Denoised with OIDN (using only the color buffer):

Here's a package with the input and output EXRs, including images rendered with unidirectional path tracing as well, and the Mitsuba scene files: mitsuba-bdpt.zip

So I'm now really puzzled why the denoiser doesn't work with LuxCore's BiDir output. Is there any significant difference between the LuxCore's and Mitsuba's BiDir implementation?

Regarding the pixel filters: I think OIDN should work with any pixel filter as long as the filter is importance sampled (e.g. it works fine for images rendered with Cycles and LuxCore). However, it really doesn't work for weighted sampling. I think this is the reason why some people reported that OIDN doesn't work with non-box filters, and others reported that it does. But some renderers support both sampling methods, e.g. RenderMan. Actually, the RenderMan documentation mentions that filter importance sampling must be used for images to be denoised.

Even though you disabled filtering in some of your images, I still find the noise pattern a bit odd. If you compare it with the Mitsuba image, it seems the noise produced by Mitsuba is much finer, but the resolution is the same. Could it be perhaps that there is still some weighted filtering going on in your images (even with a box filter with larger than 1 width)? I can't think of anything else.

[atafra]

I just found in the LuxCore BiDir implementation that FilmSampleSplatter is used to accumulate the samples, which looks pretty much weighted sampling. Is it possible that it was enabled for your "nofilter" images too (e.g. using a box filter with > 1 width)? That would explain why OIDN can't denoise it, so the issue is not related to bidirectional path tracing at all.

[atafra]

It turned out that the issue was indeed with the weighted filtering in the BiDir implementation of LuxCoreRender. It was confirmed that OIDN works well with both BiDir and Metropolis: https://forums.luxcorerender.org/viewtopic.php?f=5&t=872&start=190#p9615