search

hsahovic / poke-env

A python interface for training Reinforcement Learning bots to battle on pokemon showdown
https://poke-env.readthedocs.io/
MIT License
282 stars 93 forks source link

[Proposal] Replace `gymnasium` with `pettingzoo` #560

Open acxz opened 2 weeks ago

acxz commented 2 weeks ago

Pokemon battles are fundamentally a multi-agent system.

Currently we are using Farama's Gymnasium which is a RL env API for single-agents. I believe this decision came about as at the time of this project's creation there was no standard/popular multi-agent RL env API, leaving us to choose OpenAI's Gym, which has now moved over to Gymnasium.

However, with popularity rising in multi-agent systems, various RL env API's have been created for multi-agent systems, such as Farama's PettingZoo and Google Deepmind's OpenSpiel among others.

Overall, PettingZoo has become a strong defacto for multi-agent systems as it closely resembles Gymnasium's API, is mathematically more general than the other RL multi-agent env APIs, and more importantly has seen adoption in the popular RL libraries such as rllib, tianshou, stable-baselines3, among others.

With this in mind, and again recognizing that Pokemon battles are a multi-agent system, I believe that we should be using pettingzoo instead of gymnasium.

Practically, this would mean a change in the API for us and it should mean that certain logic we may have assumed/worked around due to using a single-agent API would be simpler in PettingZoo's API.

I'd love to hear everyone's thoughts about this and the feasibility of this change. I truly believe such a change would increase the usability of this project.

hsahovic commented 2 weeks ago

This sounds like a good idea, and I would be open to discussing a plan to implement it.