search

wyf0912 / LLFlow

[AAAI 2022] The code release of paper "AAAI Low-Light Image Enhancement with Normalizing Flow"
Other
289 stars 36 forks source link

question about the 'finetune the overall brightness' #34

Closed Boberalice closed 1 year ago

Boberalice commented 1 year ago

Hi, Thank you for your code. I used the provided pretrain model to test LOL. the average PSNR I measured is 25db, but I felt confused that when I removed the 'finetune the overall brightness', the result is 21.15. the value has a significant decline without finetuning the global brightness which uses the ground truth.

allswin commented 1 year ago

Hi, Thank you for your code. I used the provided pretrain model to test LOL. the average PSNR I measured is 25db, but I felt confused that when I removed the 'finetune the overall brightness', the result is 21.15. the value has a significant decline without finetuning the global brightness which uses the ground truth.

There are several serious problems with this paper: (1) When comparing with other methods, the same ssim calculation method is not used, which would make the SSIM of LLFLOW look higher. (2) LLFLOW uses 'finetune the overall brightness' when calculating SSIM, PSNR and LPIPS. However, the authors did not add this trick to other methods when comparing. This is not fair.

allswin commented 1 year ago

Hi, Thank you for your code. I used the provided pretrain model to test LOL. the average PSNR I measured is 25db, but I felt confused that when I removed the 'finetune the overall brightness', the result is 21.15. the value has a significant decline without finetuning the global brightness which uses the ground truth.

There are several serious problems with this paper: (1) When comparing with other methods, the same ssim calculation method is not used, which would make the SSIM of LLFLOW look higher. (2) LLFLOW uses 'finetune the overall brightness' when calculating SSIM, PSNR and LPIPS. However, the authors did not add this trick to other methods when comparing. This is not fair.

For example, the SSIM score of KIND in this paper is 0.80. However, when I use the same SSIM metric as this paper and add this trick to finetune the overall brightness, I finally find the SSIM of KIND can be as high as 0.8919!

wyf0912 commented 1 year ago

Kind/Kind++ uses the global ratio as the input of the network, and we use a simplified manner. This trick is also widely used in other papers.