RenderKit / oidn

Intel® Open Image Denoise library
https://www.openimagedenoise.org/
Apache License 2.0
1.74k stars 160 forks source link

oidnDenoiser produces an unexpected, non-denoised image #159

Closed georgy95 closed 1 year ago

georgy95 commented 1 year ago

Tested on Linux VM and a MacOS M1 chip. Issue on both.

When I denoise an image using the example oidnDenoise executable, nothing is denoised and I get my input back. However, it does work on the Mazda example listed in the README.


I save the Mazda image in 2 ways (files attached below):

  1. Saved as original raw file
  2. Took a screenshot of the Mazda image

In case 1, the image is denoised. In case 2, the image is the same as input.

Custom images

Input

mazda-screenshot

Denoised Output

mazda-screenshot-denoised

Using the example Mazda image listed in the README.md works as expected

Input

mazda-original

Output

mazda-original-denoised

PMF & JPEG input images for convenience

images.zip

What could be causing this?

atafra commented 1 year ago

This is not a bug. You're trying to denoise an image manipulated in a way that destroyed the original noise pattern. This prevents the detection of the Monte Carlo noise, thus the image remains noisy. Images should not be resampled, compressed with a lossy algorithm (e.g. JPEG), etc. before denoising. All these should be done after denoising.

georgy95 commented 1 year ago

Thanks. I see. Is there any sources for classification of Monte Carlo noise?

I discovered this issue was after rendering a 3D scene for a short period of time (10 samples) - until it was similarly noisy as the car picture. My scene had a sample object on a transparent background. I put the PNG image through the denoiser and and the noise wasn't removed.

I understand the JPEG is lossy, but a PNG screenshot (which I assume is lossless) also does not work?

atafra commented 1 year ago

Open Image Denoise is meant to be used as a library integrated into renderers, not as a stand-alone application, primarily because its correct usage requires domain knowledge and also knowledge about the internals of the renderer, most importantly the sampling approaches. Currently splatting for pixel filtering is not supported, only filter importance sampling.

It's hard to say why you still don't get a denoised image. I definitely advise against taking any screenshots because the OS might apply display scaling which would also destroy the noise pattern. It's also possible that the renderer you're using uses splatting. Or the renderer applies some post-processing effect which distorts the image, which is possibly barely visible. There can be many reasons. Storing the image in lossless format is not a guarantee in itself that denoising is possible.

georgy95 commented 1 year ago

I am using a scene rendered in Three.js - it get's pathtraced and I save the scene for denoising (as a PNG). The scene is a just a sample 3D object on a transparent background.

I'll try rendering a few more samples and hopefully get it to work.