search

zhutmost / lsq-net

Unofficial implementation of LSQ-Net, a neural network quantization framework
MIT License
276 stars 40 forks source link

Error: Divided by 0. #17

Open woriazzc opened 2 years ago

woriazzc commented 2 years ago

Thanks for your the implementation!

In LsqQuan. https://github.com/zhutmost/lsq-net/blob/2c24a96be06d044fa4c7d651727f4574b8d88c86/quan/quantizer/lsq.py#L54 If s_scale contains 0, there will be Divided by 0 Error, and the training loss will become nan. Maybe eps should be used.

zhutmost commented 2 years ago

Looks that it is very rarely seen. s_scale is a floating pointed number, it generally does not become 0 during training.

I will try to add an eps, when I have spare time. (it need some experiments)

lippman1125 commented 2 years ago

Looks that it is very rarely seen. s_scale is a floating pointed number, it generally does not become 0 during training. I will try to add an eps, when I have spare time. (it need some experiments)

Not rarely.

flymmmfly commented 11 months ago

Not rarely.