search

AntreasAntoniou / HowToTrainYourMAMLPytorch

The original code for the paper "How to train your MAML" along with a replication of the original "Model Agnostic Meta Learning" (MAML) paper in Pytorch.
https://arxiv.org/abs/1810.09502
Other
773 stars 137 forks source link

Warm restarts? #26

Closed aatkinson closed 2 years ago

aatkinson commented 4 years ago

Hi Antreas, thanks for providing this code!

Per your paper and code you use a cosine-annealed learning rate in the outer loop, but it is unclear if you also use warm restarts, as described in the SGDR paper which introduced this schedule.

Have you experimented with warm restarts at all, or am I just missing something?

brando90 commented 2 years ago

did you get an answer to this?

aatkinson commented 2 years ago

Answered in your issue here :) https://github.com/AntreasAntoniou/HowToTrainYourMAMLPytorch/issues/43 no warm restarts