Closed mcabbott closed 7 months ago
I would really like to see this through. For instance, I hate having to copy and paste the default values of beta1 and beta2 from the docstring of AdamW and write AdamW(1e-3, (0.9, 0.999), 1e-4)
instead of AdamW(eta=1e-3, decay=1e-4)
Sorry I missed your comment here. But AdamW
precisely my motivating case too. Maybe we should do it?
No objections from me, though I would put in pin in un-greeking some of the common shared hyperparam names for the next major release.
Allows this:
Not entirely sure it's worth the complexity & extra documentation.
(Note that
adjust!
already exposes field names, so in a sense they are already API.)Checklist