search

davidtvs / PyTorch-ENet

PyTorch implementation of ENet
MIT License
389 stars 129 forks source link

ENet returns a tensor with Nans as an output #20

Closed kushnir95 closed 5 years ago

kushnir95 commented 5 years ago

I tried to train ENet from scratch on my own data and after several epochs of training ENet started to return a tensor with Nan values. Can any part of the model cause this problem? If it is true, how we can handle it?

davidtvs commented 5 years ago

Something like that is more likely to be connected to faulty input images (large values, NaN, Inf) or training parameters that cause instability (high learning rate).