search

FluxML / Optimisers.jl

Optimisers.jl defines many standard optimisers and utilities for learning loops.
https://fluxml.ai/Optimisers.jl
MIT License
74 stars 22 forks source link

Add `adjust!` #113

Closed mcabbott closed 1 year ago

mcabbott commented 2 years ago

Now that Leaf is mutable, perhaps it is useful to be able to change eta only for some parts of the model.

PR also solves the following bug: adjust doesn't copy the momenta, and thus they remain shared with the previous object.

PR Checklist

mcabbott commented 1 year ago

We should do this, IMO. It fits with freeze!, and updating e.g. https://fluxml.ai/Flux.jl/dev/tutorials/2021-02-07-convnet/ to explicit parameters would like to use it.

Maybe adjust should in fact be marked deprecated, in favour of adjust!?