juglab / n2v

This is the implementation of Noise2Void training.
Other
387 stars 107 forks source link

Does it make sense: PSNR between original and de-noised image? #96

Closed JohnCraigPublic closed 4 years ago

JohnCraigPublic commented 4 years ago

Using N2V, I wanted to try to quantify and understand the 'improvement' is makes to one of my images. Does it make sense to compute PSNR between the original (noisy) image and the de-noised image? How would one interpret the PSNR value? The larger it is, the "less difference" there is between the images - so N2V "did less" to the image -- perhaps because it was already relatively low noise? A pair with low PSNR means that N2V changed the image quite a bit, perhaps meaning that original image was more noisy to start with. So, its not really telling me a metric of how well N2V did, but its giving me a metric of how noisy the original image was. Does this kind of thinking make sense?

tibuch commented 4 years ago

I would not recommend to use PSNR in this way. Mostly because people are used to PSNR as a measurement between an image and a corresponding ground truth image. In case you use it between restored and input image I would strongly recommend to give it a different name, to avoid confusion :slightly_smiling_face:

If you want to use as a measure between restored and GT image I would also recommend to test how the measure behaves for different restoration methods e.g. Gaussian smoothing with different sigmas. I would expect the behaviour you described, but I haven't looked into it and didn't verify it at all.

As an alternative you could inspect the Fourier Spectra of the input and restored image. I would expect that the high frequency amplitudes will be greatly reduce in the restored image, hence high frequency noise got removed. But again, this would have to be verified.

The best way to verify such measures is to generate some synthetic images for which you know the GT and than evaluate the measures for different amounts of noise.

Hope this helps!

JohnCraigPublic commented 4 years ago

Thanks for your helpful comments.