search

yaoyao-liu / meta-transfer-learning

TensorFlow and PyTorch implementation of "Meta-Transfer Learning for Few-Shot Learning" (CVPR2019)
https://lyy.mpi-inf.mpg.de/mtl/
MIT License
731 stars 147 forks source link

the choice of optimizer #31

Open Sword-keeper opened 4 years ago

Sword-keeper commented 4 years ago

Hi I just found that in the pretrain phase, you choose the SGD optimizer. However, in the meta-train phase, you choose the Adam optimizer. I wonder that why you choose different optimizer in the different phase?

yaoyao-liu commented 4 years ago

We choose the optimizer by empirical results. You may change the optimizer and re-run the experiments to see the difference.

Sword-keeper commented 4 years ago

hi In the torch code,I found that you set the train_aug=false in the meta-training phase.However, in the pretrain phase, you set the train_aug = true. So the train_aug is designed for the pretrain phase? I set train_aug=ture in the meta-training phase, and runned several epochs. The result lower than aug=False.

yaoyao-liu commented 4 years ago

We apply data augmentation during pre-training to solve the overfitting problem. You may also apply data augmentation during meta-training as well. Please note that you cannot apply data augmentation on the episode test (the test set for each small task).