mseminatore / ann

LIBANN is a fast, portable and easy to use neural network library written in pure ANSI-C
MIT License
4 stars 1 forks source link

bug: relu not working #22

Open mseminatore opened 1 year ago

mseminatore commented 1 year ago

Need to update backpropagation for derivative of RELU

mseminatore commented 1 year ago

See #25

mseminatore commented 10 months ago

https://machinelearningmastery.com/how-to-avoid-exploding-gradients-in-neural-networks-with-gradient-clipping/

https://ai.stackexchange.com/questions/8491/does-it-make-sense-to-apply-softmax-on-top-of-relu

https://stackoverflow.com/questions/37448557/why-are-my-tensorflow-network-weights-and-costs-nan-when-i-use-relu-activations?rq=4

https://stackoverflow.com/questions/36498127/how-to-apply-gradient-clipping-in-tensorflow

https://www.tensorflow.org/api_docs/python/tf/compat/v1/train/Optimizer#processing_gradients_before_applying_them