Open tristandeleu opened 9 years ago
The backpropagation still occasionally outputs NaN, even with #9 fixed.
This is not an issue with the use of rectify and hard_sigmoid activations as it also appeared in https://github.com/snipsco/nlp-neural-turing-machine/issues/10#issuecomment-149141728 with activations tanh and sigmoid.
rectify
hard_sigmoid
tanh
sigmoid
The backpropagation still occasionally outputs NaN, even with #9 fixed.
This is not an issue with the use of
rectifyandhard_sigmoidactivations as it also appeared in https://github.com/snipsco/nlp-neural-turing-machine/issues/10#issuecomment-149141728 with activationstanhandsigmoid.