search

Li-Chongyi / Zero-DCE_extension

337 stars 61 forks source link

[nan loss occur] #2

Open CaptainEven opened 3 years ago

CaptainEven commented 3 years ago

Hi, nan loss occured when training on some images. How to solve this?

Li-Chongyi commented 3 years ago

try to decrease your learning rate. Or please check the training data.

Even @.***> 于2021年7月20日周二 上午10:31写道：

Hi, nan loss occured when training on some images. How to solve this?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Li-Chongyi/Zero-DCE_extension/issues/2, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIEVQEJD7P5VMBGJFPRJ2Z3TYTNY5ANCNFSM5AU3IWVA .

senlin-ali commented 2 years ago

Hi, nan loss occured when training on some images. How to solve this?

hi i have the same question and have you solved it

Wuwang365 commented 2 years ago

Hi, the same question, has anyone solved it?

Zruto commented 2 years ago

same question

Schizophreni commented 2 years ago

This may be caused by the color loss defined in L_color class (line 9 in Myloss.py), where torch.pow(xx+xx+xx, 0.5) is used. Note that the gradient of sqrt(x) is 1/2*sqrt(x), which is not derivable when x=0. Therefore by changing torch.pow(xx+xx+xx, 0.5) to torch.pow(xx+xx+xx+1e-5, 0.5), this problem is solved in my case. However, the color loss can be modified as is written in the paper. For example, k= (mr-mb)2+(mr-mg)2+(mb-mg)**2 .