Solver

[dmccloskey]

Description

Abstract base and inherited classes for various solvers

Objectives

• [x] SGD
• [ ] Gradient noise

Objectives (Bonus)

• [x] Adam
• [ ] NAdam
• [ ] AMSGrad

References

• Neelakantan, A., Vilnis, L., Le, Q. V., Sutskever, I., Kaiser, L., Kurach, K., & Martens, J. (2015). Adding Gradient Noise Improves Learning for Very Deep Networks, 1–11. Retrieved from http://arxiv.org/abs/1511.06807 ↩
• http://ruder.io/optimizing-gradient-descent/

[dmccloskey]

http://caffe.berkeleyvision.org/tutorial/solver.html