AntreasAntoniou
commented
4 years ago The best way to do this would be to increase the steps at training time to reflect the setup you want for testing time. If you must train for less steps than you test, then try the following:
- Learn a learning rate for each layer for all the steps (i.e. having only 1 learning rate for each layer instead of 5).
- Replace batch norm with a normal, inner loop adaptable batch norm.
Then it should be fine.
On Fri, 27 Mar 2020 at 01:14, Big-Wuu notifications@github.com wrote:
Dear authors, Your ideas to deal with the instability of MAML is great and elegant. But there is a simple but vital question with Per-step BN and Per-step LR, that's can they deal with the situation when the number of inner-loop steps at test time is different from training time? e.g. I only update 5 times during training but I want to update more times at test. It seems can't because we don't have the statistics out of range.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/AntreasAntoniou/HowToTrainYourMAMLPytorch/issues/28, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACSK4NVQ4RYMUJB3Q7N4XHDRJP4XHANCNFSM4LUU63GA .
Dear authors, Your ideas to deal with the instability of MAML is great and elegant. But there is a simple but vital question with Per-step BN and Per-step LR, that's can they deal with the situation when the number of inner-loop steps at test time is different from training time? e.g. I only update 5 times during training but I want to update more times at test. It seems can't because we don't have the statistics out of range.