search

FluxML / Optimisers.jl

Optimisers.jl defines many standard optimisers and utilities for learning loops.
https://fluxml.ai/Optimisers.jl
MIT License
72 stars 20 forks source link

Add all-keyword constructors, much like `@kwdef` #160

Closed mcabbott closed 7 months ago

mcabbott commented 12 months ago

Allows this:

julia> AdamW(lambda=333e-3)
OptimiserChain(Adam(0.001, (0.9, 0.999), 1.0e-8), WeightDecay(0.333))

Not entirely sure it's worth the complexity & extra documentation.

(Note that adjust! already exposes field names, so in a sense they are already API.)

Checklist

CarloLucibello commented 11 months ago

I would really like to see this through. For instance, I hate having to copy and paste the default values of beta1 and beta2 from the docstring of AdamW and write AdamW(1e-3, (0.9, 0.999), 1e-4) instead of AdamW(eta=1e-3, decay=1e-4)

mcabbott commented 11 months ago

Sorry I missed your comment here. But AdamW precisely my motivating case too. Maybe we should do it?

ToucheSir commented 11 months ago

No objections from me, though I would put in pin in un-greeking some of the common shared hyperparam names for the next major release.