Tf 0.3.0 - Githubissues

juntang-zhuang / Adabelief-Optimizer

Repository for NeurIPS 2020 Spotlight "AdaBelief Optimizer: Adapting stepsizes by the belief in observed gradients"

BSD 2-Clause "Simplified" License

1.05k stars 108 forks source link

Tf 0.3.0 #39

Closed cryu854 closed 3 years ago

cryu854 commented 3 years ago

Update notes:

Implement an adabelief in tf1.x version.
- The optimizer inherits from tf.train.Optimizer and supports same features as adabelief in tf2.
Add test code for adabelief in tf1 and tf2.
- The code was developed from tensorflow's adam_test.py.
Optimize rectify.
- For adabelief in tf2, some calculations are not being used when rectify = False.

I also tried several examples from https://github.com/aymericdamien/TensorFlow-Examples/tree/master/tensorflow_v1/notebooks for tf1.x version.

juntang-zhuang commented 3 years ago

@cryu854 Thanks a lot. Will merge it tomorrow. Happy new year!