huawei-noah / Pretrained-IPT

Apache License 2.0
424 stars 63 forks source link

For all tasks, PSNR is calculated only for y, rather than RGB. #48

Closed DPS2022 closed 1 year ago

DPS2022 commented 1 year ago

Thank you for the nice work and for sharing the pre-trained model!

I found an error in the implementation of PSNR calculation.

"For all tasks, PSNR is calculated only for y, rather than RGB values."

The below function has "y" as the default argument for cal_type, and for all tasks, no argument for cal_type is given.

https://github.com/huawei-noah/Pretrained-IPT/blob/a8f63ddda41498b38c5322be48308dc8d005e526/utility.py#L168-L184

For example, this is the way to calculate PSNR for denoising task.

https://github.com/huawei-noah/Pretrained-IPT/blob/a8f63ddda41498b38c5322be48308dc8d005e526/trainer.py#L63-L75

If we use cal_type="y", it gives 32.415 dB for gaussian denoising (simga=30) with CBSDS68 dataset.

But, if we give the other argument, it gives 30.748 dB which is much lower than the reported value in the paper.

Furthermore, if we save the result and calculate PSNR using skimage library, it gives around 30.75 dB, too.

Could you check the PSNR values?

HantingChen commented 1 year ago

Sorry for the misleading, we calculate PSNR in y channels for all tasks. If you give other argument, the code will calculate PSNR in RGB channel, which is much lower than the results calculated in y channel

DPS2022 commented 1 year ago

Ah, got it. Thanks for reply :)