Feature request: Include the Adam algorithm

JuliaNLSolvers / Optim.jl

Optimization functions for Julia

Other

1.11k stars 213 forks source link

Feature request: Include the Adam algorithm #1012

Closed renatobellotti closed 5 months ago

renatobellotti commented 1 year ago

Hi,

I would like to suggest including the Adam algorithm into Optim.jl. Adam is used extensively in the deep learning community, where it is the go-to algorithm. I found a small implementation: https://gist.github.com/vankesteren/96207abcd16ecd01a2491bcbec12c73f

pkofod commented 1 year ago

I have Adam and AdaMax here https://github.com/JuliaNLSolvers/NLSolvers.jl

renatobellotti commented 1 year ago

Nice, thank you! Is it already compatible with the Optim.jl API? I couldn't find it in the documentation of Optim.jl... Would be great to have more options to experiment with. :)

pkofod commented 5 months ago

Sorry. I never had a need for this, but since it was requested again, I made a PR here though it may not longer be relevant for you :) https://github.com/JuliaNLSolvers/Optim.jl/pull/1069