search

FluxML / Optimisers.jl

Optimisers.jl defines many standard optimisers and utilities for learning loops.
https://fluxml.ai/Optimisers.jl
MIT License
74 stars 22 forks source link

2nd Order Optimi z/s ers #4

Open samiit opened 4 years ago

samiit commented 4 years ago

Hi, I am new to this ecosystem and was exposed to it via the talk by Dhairya at JuliaCon2020. I thought to ask if there are any plans for you to include 2nd order Optimizers, that were recently introduced by the very recent work on AdaHessian. Kindly share if it is already included and if there are any benchmarks as to how they perform compared to the 1st order optimizers! Thanks.

Sam

freemin7 commented 3 years ago

Another optimizer that uses higher order information is called Ridge Rider which might be really interesting to implement

darsnack commented 3 years ago

Now that #16 is merged, we could support higher order rules if someone wanted to write them.