sjvasquez / web-traffic-forecasting

Kaggle | Web Traffic Forecasting 📈
666 stars 239 forks source link

During training, train loss and validation loss became nan, does this matter? #4

Open nce3xin opened 5 years ago

nce3xin commented 5 years ago

When I ran cnn.py, during training, train loss and validation loss became nan after step 50, is this normal? I wonder why losses remains nan……

leonJCY commented 4 years ago

Me too, Do you use gpu ? I had this problem when I use CPU to train.

leonJCY commented 4 years ago

If you use gpu, it is 2080ti for me, then all the problems will be gone

wentixiaogege commented 3 years ago

10 step becaome nan.. using cpu~~~

wentixiaogege commented 3 years ago

change lr ,10 times smaller , work~;