search

kengz / SLM-Lab

Modular Deep Reinforcement Learning framework in PyTorch. Companion library of the book "Foundations of Deep Reinforcement Learning".
https://slm-lab.gitbook.io/slm-lab/
MIT License
1.23k stars 264 forks source link

Evolution Strategies and Genetic Algorithms Policy in DRL #459

Open bturkoglu opened 4 years ago

bturkoglu commented 4 years ago

thank you so much for nice job. I want to implement one of the algorithms without gradient in this project and compare the results with the algorithms in this project such as actorcritic, dqn ,reinforce.

I have a code that works in Pytorch https://towardsdatascience.com/reinforcement-learning-without-gradients-evolving-agents-using-genetic-algorithms-8685817d84f.

Deep Neuroevolution: Genetic Algorithms are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning https://arxiv.org/pdf/1712.06567.pdf

Evolution Strategies as a Scalable Alternative to Reinforcement Learning https://arxiv.org/pdf/1703.03864.pdf

How can I do the implementation?

kengz commented 4 years ago

Hi @bahaTRKGLU I have looked at evolutionary methods a little, but the main challenge is the API. The lab is able to implement all the algorithms within a shared framework because they have a common API - the control loop here.

This means for any implementation, it needs to obey the agent API. A simple example that's non-gradient is the random agent. If you're able to make evolutionary method conform to this API then u can directly plug and play it in the lab.

Interested to see if you have a design in mind!

bturkoglu commented 4 years ago

I basically want to implement the code shared by uber and compare the results with the algorithms in SLM-lab. But I'm a rookie in this regard and I couldn't. Can you add this algorithm to SLM-lab at a convenient time?

kengz commented 4 years ago

Unfortunately there's no plan to do so and we the authors are quite occupied, but I'll mark this as help wanted for anyone who wishes to take it on.