search

wyf0912 / LLFlow

[AAAI 2022] The code release of paper "AAAI Low-Light Image Enhancement with Normalizing Flow"
Other
289 stars 36 forks source link

自制数据集 nll PSNR SSIM出现nan值 #47

Closed JayaoZheng closed 4 months ago

JayaoZheng commented 8 months ago

您好，我最近用自制训练集进行训练，在114epoch nll出现异常 log信息如下： 23-12-29 14:24:01.086 - INFO: <epoch:112, iter: 11,400> psnr: 2.1525e+01 SSIM: 6.2814e-01 23-12-29 14:31:52.237 - INFO: <epoch:114, iter: 11,600> psnr: nan SSIM: nan

我参考之前类似的问题，重新训练了好几次都是在120epoch左右出现这个异常，请问有什么其它的办法吗？

wyf0912 commented 8 months ago

Hi, thanks for your interest in our work.

Maybe you could try the following work to alleviate this problem

https://arxiv.org/pdf/2212.04319.pdf

Besides, a trivial way may be that you could try to use a smaller learning rate.

wyf0912 commented 4 months ago

https://github.com/smhongok/robustNF

Thanks.

Hi, thanks for your interest in our work.

Maybe you could try the following work to alleviate this problem

https://arxiv.org/pdf/2212.04319.pdf

Besides, a trivial way may be that you could try to use a smaller learning rate.