[Proposal] Replace `gymnasium` with `pettingzoo`

[acxz]

Pokemon battles are fundamentally a multi-agent system.

Currently we are using Farama's Gymnasium which is a RL env API for single-agents. I believe this decision came about as at the time of this project's creation there was no standard/popular multi-agent RL env API, leaving us to choose OpenAI's Gym, which has now moved over to Gymnasium.

However, with popularity rising in multi-agent systems, various RL env API's have been created for multi-agent systems, such as Farama's PettingZoo and Google Deepmind's OpenSpiel among others.

Overall, PettingZoo has become a strong defacto for multi-agent systems as it closely resembles Gymnasium's API, is mathematically more general than the other RL multi-agent env APIs, and more importantly has seen adoption in the popular RL libraries such as rllib, tianshou, stable-baselines3, among others.

With this in mind, and again recognizing that Pokemon battles are a multi-agent system, I believe that we should be using pettingzoo instead of gymnasium.

Practically, this would mean a change in the API for us and it should mean that certain logic we may have assumed/worked around due to using a single-agent API would be simpler in PettingZoo's API.

I'd love to hear everyone's thoughts about this and the feasibility of this change. I truly believe such a change would increase the usability of this project.

[hsahovic]

This sounds like a good idea, and I would be open to discussing a plan to implement it.

[caymansimpson]

@acxz did you ever start/try this? Wondering about this myself

[acxz]

Nope never started actually implementing it. Knock yourself out.