search

tristandeleu / pytorch-maml-rl

Reinforcement Learning with Model-Agnostic Meta-Learning in Pytorch
MIT License
827 stars 158 forks source link

Are benchmarks available? #29

Closed quanvuong closed 5 years ago

quanvuong commented 5 years ago

Thank you for the repo!

I was wondering if the repo obtains the same performance as reported in the papers on the DRL benchmarks?

tristandeleu commented 5 years ago

Thanks for the kind words!

For the benchmarks, unfortunately I don't have anything "official" as a point of comparison with the original paper. From experiment with HalfCheetahVel and HalfCheetahDir, the results after 1000 meta-training updates seem to match the results from the original paper. There is unfortunately an issue with the Ant tasks (AntDir and AntVel), for which I have never been able to get good results (there is another related issue #25 ), and never been able to match the performance from the original repo.

I think @katerakelly might have used this code in her latest paper for the MAML-TRPO baseline. If that's the case (pinging @katerakelly just in case), then this paper also includes other experiments on Humanoid and Walker, and you could probably expect similar performances as in Figure 9.

But I think it would be a great idea to have a more principled benchmark as part of the repo for comparison with the original paper.

katerakelly commented 5 years ago

We actually did not end up using this repo for our MAML-TRPO experiments. We ended up using the implementation in https://github.com/jonasrothfuss/ProMP since that has ProMP also.

quanvuong commented 5 years ago

Thank you all for your comments! They’re super helpful.

tristandeleu commented 5 years ago

That's good to know, thank you @katerakelly!