search

dnouri / nolearn

Combines the ease of use of scikit-learn with the power of Theano/Lasagne
http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/
MIT License
949 stars 260 forks source link

Convolutional Autoencoder NaN #275

Closed sampepose closed 8 years ago

sampepose commented 8 years ago

Hi! I'm trying to make a convolutional autoencoder based off of VGG-S (https://github.com/Lasagne/Recipes/blob/master/modelzoo/vgg_cnn_s.py).

For some reason, learning always converges to NaN almost immediately. I think my architecture is correct from VGG-S, so I'm not sure why this is happening.

screenshot 2016-06-09 10 07 57

Here's my code (https://gist.github.com/sampepose/ccb58557271cff10d182f4ab8282f3b4).

BenjaminBossan commented 8 years ago

Hi. You should definitely have another nonlinearity in the last layer. My suggestion is to use sigmoid and to normalize the target to [0, 1]. If that doesn't help, I would play with different learning rates.

dnouri commented 8 years ago

I'd be curious to see what the architecture looks like. And if Op ever figured out what was wrong with his example. @BenjaminBossan's advices likely to help

Theano has documentation on Dealing with NaNs which should also help.