Added meta_lr_final parameter to indicate the final value of the meta learning rate. By setting meta_lr_final to a smaller value than meta_lr, the meta learning rate is effectively decayed during training. Before it was constant due to the bug of using meta_lr both times in the weighted sum.
Description
Fixes #[ISSUE NUMBER]
Replace this line by a one sentence summary of your PR.
If necessary, use the following space to provide context or more details.
Contribution Checklist
If your contribution modifies code in the core library (not docs, tests, or examples), please fill the following checklist.
[ ] My contribution is listed in CHANGELOG.md with attribution.
[x] My contribution modifies code in the main library.
[ ] My modifications are tested.
[ ] My modifications are documented.
Optional
If you make major changes to the core library, please run make alltests and copy-paste the content of alltests.txt below.
Added meta_lr_final parameter to indicate the final value of the meta learning rate. By setting meta_lr_final to a smaller value than meta_lr, the meta learning rate is effectively decayed during training. Before it was constant due to the bug of using meta_lr both times in the weighted sum.
Description
Fixes #[ISSUE NUMBER]
Replace this line by a one sentence summary of your PR.
If necessary, use the following space to provide context or more details.
Contribution Checklist
If your contribution modifies code in the core library (not docs, tests, or examples), please fill the following checklist.
Optional
If you make major changes to the core library, please run
make alltests
and copy-paste the content ofalltests.txt
below.