Nan Loss during training MNIST dataset

brjathu / iTAML

Official implementation of "iTAML : An Incremental Task-Agnostic Meta-learning Approach". CVPR 2020

95 stars 16 forks source link

Nan Loss during training MNIST dataset #10

Closed mmderakhshani closed 3 years ago

mmderakhshani commented 4 years ago

Hi. First of all, I have to appreciate regarding your great implementation.

During training on MNIST dataset using the following command, I found nan: Screenshot (395)

Could you please explain to me am I doing something wrong? I just ran following command:

python train_mnist.py 0

brjathu commented 4 years ago

It seems a problem in the code. I will check on that, meanwhile, You can stop after getting NAN, and run python train_mnist.py 3 it will start from session 3 and after the restart, it seems working fine.

mmderakhshani commented 4 years ago

Thanks for the reply and thanks for the suggestion.