search

garlicbulb-puzhuo / ggo

1 stars 0 forks source link

Debug training divergence on both theano/tensorflow #75

Closed tomowind closed 7 years ago

oakcreek commented 7 years ago

Solution 1: try increase optimizer learning rate. See branch '20170304'.

tomowind commented 7 years ago 
$ git diff master
diff --git a/scripts/optimizer.py b/scripts/optimizer.py
index 27e381c..33ea871 100644
--- a/scripts/optimizer.py
+++ b/scripts/optimizer.py
@@ -1,4 +1,4 @@
 from keras.optimizers import Adam, SGD

 sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True, clipnorm=1.0)
-adam = Adam(clipnorm=1.0)
+adam = Adam(lr=0.01, clipnorm=1.0)
tomowind commented 7 years ago

See #79