search

GuoShi28 / CBDNet

Code for "Toward Convolutional Blind Denoising of Real Photographs", CVPR 2019
Apache License 2.0
499 stars 92 forks source link

The results in Table 2 are strange #22

Open csjunxu opened 5 years ago

csjunxu commented 5 years ago

In Table 2, the PSNR and SSIM results on the 15 cropped images provided by Nam et al. in CVPR 2016 are not consistent with the paper of Nam et al. (CVPR 2016), MCWNNM, TWSC, the method of NI (neat image software). How do you compute the PSNR and SSIM for Table 2?

Here are my PSNR results:

NI & CC & MCWNNM & TWSC & DnCNN+ &FFDNet+&CBDNet 35.68 & 38.37 & 41.13 & 40.76 & 38.02 & 39.35 & 36.68
34.03 & 35.37 & 37.28 & 36.02 & 35.87 & 36.99 & 35.58
32.63 & 34.91 & 36.52 & 34.99 & 35.51 & 36.50 & 35.27
31.78 & 34.98 & 35.53 & 35.32 & 34.75 & 34.96 & 34.01
35.16 & 35.95 & 37.02 & 37.10 & 35.28 & 36.70 & 35.19
39.98 & 41.15 & 39.56 & 40.90 & 37.43 & 40.94 & 39.80
34.84 & 37.99 & 39.26 & 39.23 & 37.63 & 38.62 & 38.03
38.42 & 40.36 & 41.43 & 41.90 & 38.79 & 41.45 & 40.40 35.79 & 38.30 & 39.55 & 39.06 & 37.07 & 38.76 & 36.86 38.36 & 39.01 & 38.91 & 40.03 & 35.45 & 40.09 & 38.75 35.53 & 36.75 & 37.41 & 36.89 & 35.43 & 37.57 & 36.52 40.05 & 39.06 & 39.39 & 41.49 & 34.98 & 41.10 & 38.42 34.08 & 34.61 & 34.80 & 35.47 & 31.12 & 34.11 & 34.13 32.13 & 33.21 & 33.95 & 34.05 & 31.93 & 33.64 & 33.45 31.52 & 33.22 & 33.94 & 33.88 & 31.79 & 33.68 & 33.45 Average 35.33 & 36.88 & 37.71 & 37.81 & 35.40 & 37.63 & 36.44

GuoShi28 commented 5 years ago

Hi, (1) I test 25 randomly chosen patches of Nam. These patches are provided in the testset folder. (2) The MCWNNM and TWSC codes are directly downloaded from the Github page. And I run the code on these patches without changing any parameters. (3) CBDNet(JPEG) model is recommended to test on JPEG images. Original CBDNet model may not perform start-of-the-art since noisy images in Nam are stored after JPEG compression. CBDNet model did not consider this aspect. (4) I am also surprised that WNNM and BM3D can perform really well on Nam. I gauss this is because the noise level in Nam is quite low, so even they did not consider the real-world noise distribution, they still obtain a not-bad psnr value.

csjunxu commented 5 years ago

Thank you for your detailed answer!

The key difference among the results is that, you did not test on the default 15 cropped images provided by Nam et al. in CVPR 2016. Instead, you cropped 25 images from their 11 large images and test on the 25 images. And yes, it is not your job to tune the parameters of other methods.