search

tristandeleu / pytorch-maml-rl

Reinforcement Learning with Model-Agnostic Meta-Learning in Pytorch
MIT License
827 stars 158 forks source link

Could u kindly implement pytorch-maml part? #7

Closed dragen1860 closed 6 years ago

dragen1860 commented 6 years ago

Hi, I found it's very difficult to implement the code in MAML. Currently my implementation has some bugs: https://github.com/dragen1860/MAML-Pytorch Im not sure how to do 2nd derivate in MAML. And also I search other implementation in pytorch and there are more severe bugs in them.

So would u kindly publish a correct verion of pytorch-maml or You CAN just review my implementation above and help me see any bugs? Thanks so much.

dendisuhubdy commented 6 years ago

@dragen1860 that is something pretty rude to say.

dragen1860 commented 6 years ago

Sorry for that. But i dnt understand why. I mean pytorch-maml NOT your pytorch-maml-rl. @dendisuhubdy

tristandeleu commented 6 years ago

Hi @dragen1860 , I'm sorry I haven't worked on the supervised setting of MAML unfortunately, so I wouldn't be of much help here I guess. Check the way policies are implemented to get some inspiration on how to handle the parameter adaptation on the learner (which I guess would be similar in the supervised case). The second derivative is computed automatically through the automatic differentiation of Pytorch (no specific tricks required, other than create_graph=True when computing the gradient for the adaptation).

I also have seen this implementation which seems popular, but I haven't tried it myself. Hope this will help you!