elliottower
commented
9 months ago Thanks for asking, there's a number of ones I'm aware of at least--there are surely more options out there.
I can see it being a bit confusing how the tutorials are structured, would having a centralized tutorials page be helpful to you?
I was thinking of just making a training tutorials landing page and giving some high level "here's where you should start and your options" as well as making a list of "third party tutorials", similar to https://gymnasium.farama.org/main/tutorials/third-party-tutorials/ (for example, newly released Cogment Lab which we can't officially endorse but would love to provide links to the tutorials for)
But for your question though, SB3 is the only single agent RL library in our tutorials. LangChain was largely added just as a "look at this cool thing you can do watching GPT-4 try to play chess" but would arguably be better off in the third-party tutorials page. It may also incentivize users in the same way the third-party-environments page does--it's nice to be able to show off your environment or tutorial.
- CleanRL is mainly single agent but it does have a tutorial for doing multi agent PPO, which I have updated and included in our documentation and we have our own simpler implementation (written by @jjshoots) https://pettingzoo.farama.org/tutorials/cleanrl/
- RLlib is the most fully featured I'm aware of (see https://docs.ray.io/en/latest/rllib/rllib-algorithms.html), but it has a lot of complexity and a bit of a learning curve, more industry oriented than research (disclaimer: their software is free but it is part of Anyscale which is a for-profit company) https://pettingzoo.farama.org/tutorials/rllib/
- AgileRL has pretty active development and implements a few different algorithms as well as self play, they're a good balance of ease of use and performance (disclaimer: their software is free but it is a for profit company) https://pettingzoo.farama.org/tutorials/agilerl/
- SB3 I just added as tons of people requested it, and there was this old medium article by the original creator of PettingZoo that was popular, the tutorials I made basically updated that in order to work with current version and SB3, but it's very much just a proof of concept and not the most polished thing https://towardsdatascience.com/multi-agent-deep-reinforcement-learning-in-15-lines-of-code-using-pettingzoo-e0b963c0820b
jkterry1
Question
I checked the tutorials on your website, only to find some single-agent RL algorithm libraries (SB3, Clean RL, etc). If any, what MARL libraries would you recommend when adapting my custom PettingZoo environment?