search

openai / train-procgen

Code for the paper "Leveraging Procedural Generation to Benchmark Reinforcement Learning"
https://openai.com/blog/procgen-benchmark/
MIT License
169 stars 55 forks source link

Request release of the code for Rainbow experiments #10

Closed kaixin96 closed 4 years ago

kaixin96 commented 4 years ago

Thanks for the great benchmark. Can you help release the code for experiments with Rainbow? Thank you.

Best regards

christopherhesse commented 4 years ago

I'm afraid we do not have plans to release that code currently. You can check out Dopamine which is a pretty official rainbow implementation as well as the rainbow hyperparameters from the procgen benchmark paper in section D.2.

kaixin96 commented 4 years ago

@christopherhesse May I ask if you use ape-x since the original Rainbow is designed to run on single-GPU? Does each worker maintain their own replay buffer?

I am trying to reproduce the Rainbow results. Any advice on implementations is very appreciated.

Thank you and best regards!

christopherhesse commented 4 years ago

It sounds like we used anyrl-py's implementation, and each worker has its own independent replay buffer. The parameters are synchronized using baselines MpiAdamOptimizer to provide the communication between the GPUs.

kaixin96 commented 4 years ago

Thanks for your quick reply !