search

AntreasAntoniou / HowToTrainYourMAMLPytorch

The original code for the paper "How to train your MAML" along with a replication of the original "Model Agnostic Meta Learning" (MAML) paper in Pytorch.
https://arxiv.org/abs/1810.09502
Other
759 stars 137 forks source link

Questions about new networks #8

Closed buaaswf closed 5 years ago

buaaswf commented 5 years ago

Hi @AntreasAntoniou , I want to apply the maml or mal++ learning to https://github.com/huanglianghua/siamfc-pytorch/blob/master/siamfc.py, cloud you please give me some suggestions?

AntreasAntoniou commented 5 years ago

Can you provide more information on what you are trying to achieve? The siamese networks you have linked are framed in a different learning paradigm. You'd have to re-frame the Siamese Networks for a meta-learning setting first. Then, we can talk about how you can go about applying the ideas in MAML++ in it.

buaaswf commented 5 years ago

Dear @AntreasAntoniou , Thank you for your help. I try to re-frame the Siamese Networks to fit for the few shot learning. I just want to train the learning rate for this network. I think there is only one parameter for this frame, so it will not be so hard. Could I only train the learning rate with maml?

AntreasAntoniou commented 5 years ago

Training the learning rates for such a system would almost certainly require jointly training the parameter initialization as well. If you were to only learn learning rates, they would only work well to the particular random init your model had, and would likely not generalize to any other initializations. Now, if you chose to randomly re-init the model at each iteration, so you could learn an init agnostic configuration for the learning rate, then your learning rates would only generalize for the first N steps of training, and not any further.

That being said, I'll need very specific information on the project you have in mind before I can meaningfully help you out. If you write up a detailed proposal of what you want to do and send it to me, then, I would be able to help you out.