learnables / learn2learn

A PyTorch Library for Meta-learning Research
http://learn2learn.net
MIT License
2.61k stars 350 forks source link

Meta-learning rate in reptile example is not decaying (as of now). #343

Closed mi92 closed 2 years ago

mi92 commented 2 years ago

Added meta_lr_final parameter to indicate the final value of the meta learning rate. By setting meta_lr_final to a smaller value than meta_lr, the meta learning rate is effectively decayed during training. Before it was constant due to the bug of using meta_lr both times in the weighted sum.

Description

Fixes #[ISSUE NUMBER]

Replace this line by a one sentence summary of your PR.

If necessary, use the following space to provide context or more details.

Contribution Checklist

If your contribution modifies code in the core library (not docs, tests, or examples), please fill the following checklist.

Optional

If you make major changes to the core library, please run make alltests and copy-paste the content of alltests.txt below.

[PASTE HERE]
seba-1511 commented 2 years ago

Thanks for the PR @mi92 -- the issues blocking the merge were resolved upstream so I'm merging this now.