search

deeplearning4j / deeplearning4j

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learn...
http://deeplearning4j.konduit.ai
Apache License 2.0
13.6k stars 3.83k forks source link

Use clip_by_norm for updating gradients #4384

Open AlexDBlack opened 6 years ago

AlexDBlack commented 6 years ago

Not a high priority to replace this, but clip_by_norm is now a thing in ND4J that could be used here: https://github.com/deeplearning4j/deeplearning4j/blob/80260f8c86a47a2821b771a23465595d310c8a46/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/updater/BaseMultiLayerUpdater.java#L356-L373

raver119 commented 6 years ago

Once new CustomOps get CUDA support - this issue should be implemented.