cifar_ard loss is nan. - Githubissues

HolyBayes / pytorch_ard

Pytorch implementation of Variational Dropout Sparsifies Deep Neural Networks

MIT License

83 stars 16 forks source link

cifar_ard loss is nan. #2

Closed toshi2k2 closed 4 years ago

toshi2k2 commented 5 years ago

Ran the cifar_ard code - the train and test loss shows nan around 6th epoch. Are there any issues with the code?

HolyBayes commented 4 years ago

Hi! Such an issue takes a place because of wrong initialization of sigmas. Try to initialize it with a smaller values. I'll try to use more flexible initialization of it soon

HolyBayes commented 4 years ago

Fixed