search

omimo / xRBM

Implementation of Restricted Boltzmann Machine (RBM) and its variants in Tensorflow
https://omid.al/projects/xRBM.html
MIT License
55 stars 22 forks source link

nan value #1

Closed YCWang6 closed 7 years ago

YCWang6 commented 7 years ago

Hi,

I am new to Neural Network. Thanks a lot for your xRBM! However, when I try the rbm_mnist_simple.py, nan values are created (sometimes, not always) when I set num_hid to 2. The printed massage is: Epoch 2/15|cost=nan|lr=0.100000|monentum=0.000000|sparse cost=0.000000. Would you please take a look at this issue? Thank you very much!

regards, YC

omimo commented 7 years ago

Hey YC,

Using only 2 hidden units for the MNIST dataset seems a bit too small. Try using at least 50-100 units. Also, you might get NaNs if you use a too big learning rate. You can also try 0.01 or 0.001 for the lr.

Cheers, Omid