cogment / cogment-verse

Research platform for Human-in-the-loop learning (HILL) & Multi-Agent Reinforcement Learning (MARL)
https://cogment.ai/cogment_verse
Apache License 2.0
78 stars 15 forks source link

Feature request: update PettingZoo version #142

Closed elliottower closed 1 year ago

elliottower commented 1 year ago

Hi, would it be possible to update this repo to use the most recent version of PettingZoo? We want to list this project in PettingZoo's third-party-environments, but we can only include environments which work with the current version.

If you need any help working out issues due to different versions feel free to ask, there were some breaking changes in version 1.2, so it requires a bit of code changes to adapt. The previous API returned done in the step() function, whereas the new one returns truncated and terminated (matching gymnasium). There is a migration guide for gymnasium explaining the changes further, the steps should be basically the same (we're working on making resources for updating old PettingZoo repositories as well): https://gymnasium.farama.org/content/migration-guide/

saikrishna-1996 commented 1 year ago

Thank you for bringing this up. We are looking into it!

elliottower commented 1 year ago

Thank you for bringing this up. We are looking into it!

Sounds good, reference this issue or mention here if you end up updating or have any questions

wduguay-air commented 1 year ago

Hi @elliottower, I just took a jab at bumping the version. The issue we are facing is that we need to both support:

for environments used by our users (some are still using the not maintained gym). However, some gym extras are causing dependency issues. Notably with requirements on pygame version.

Snippet from the setup.py:

"box2d": ["box2d-py==2.3.5", "pygame==2.1.0", "swig==4.*"],
"classic_control": ["pygame==2.1.0"],

And the latest versions of PettingZoo require pygame==2.3.0 Have you experience this issue with other projects using PettingZoo and did you have a solution?

elliottower commented 1 year ago

The problem is using gym rather than gymnasium, dependencies for gym will be old and incompatible with current PettingZoo or Gymnasium. But a solution to this is done in our Shimmy and ViZDoom projects, which support both Gymnasium and Gym, using optimal requirements (pip install shimmy[gym-v21])

edit: checked locally to try and reproduce your error and was able to install the requirements successfully with the new PettingZoo/Gymnasium versions

elliottower commented 1 year ago

I see in these two PRs https://github.com/cogment/cogment-verse/pull/173 https://github.com/cogment/cogment-verse/pull/135 updated the requirements and installation instructions, I'm thinking of doing a PR updating the PettingZoo and Gymnasium versions and fixing any related issues.

Also I noticed the instructions for PettingZoo Atari with M1/M2 apple silicon, any chance someone could help make a PR to the multi-agent-ale to help fix this? I don't have access to the hardware to test this myself, but it would be good to get fixed, so that PettingZoo can be fully installed on all platforms

elliottower commented 1 year ago

Another minor thing is that PettingZoo is stylized as a single word with capitalization, rather than petting zoo as it is used in some of the documentation, I can update the usages of it to be consistent in my PR

wduguay-air commented 1 year ago

@lhnguyen102 Would you be able to help with the M1/M2 apple silicon platform issues?

wduguay-air commented 1 year ago

After removing gym entirely from our requirements and replacing it with Gymnasium, I faced this other issue now between PettingZoo and Gymnasium optional dependencies. I opened a proposal for Gymnasium: https://github.com/Farama-Foundation/Gymnasium/issues/550

lhnguyen102 commented 1 year ago

@elliottower For M1/M2, I have fixed the issue related to the build last December 2022. My PR is merged to main on Multi-Agent-ALE repo. We are waiting for the PyPI release

wduguay-air commented 1 year ago

Will finish the update once Gymnasium releases v0.29 to pypi. gym will be removed from dependencies in favor of the latest versions of Gymnasium and Petting Zoo.

elliottower commented 1 year ago

Will finish the update once Gymnasium releases v0.29 to pypi. gym will be removed from dependencies in favor of the latest versions of Gymnasium and Petting Zoo.

@pseudo-rnd-thoughts (not sure if you can tag from other repost can confirm but I know gymnasium 0.28.2 is being released in the next week or so, but after that I believe they were going to aim for 1.0 release which won’t be for a while.

pseudo-rnd-thoughts commented 1 year ago

Hey, the plan which could change is a v0.28.2 then a v0.29 to deprecate features then a v1.0alpha to let users test without the deprecated features and several other changes. The timeline is not certain but I'm hoping for all of this in the next month

cloderic commented 1 year ago

This is blocked waiting for release of the versions

pseudo-rnd-thoughts commented 1 year ago

Apologies for the wait, we are not doing the v0.28.2 release anymore, we are still working on v0.29 and hoping to get it out as soon as possible

wduguay-air commented 1 year ago

Integrated changes with Gymnasium v0.29 Will wait for the PettingZoo release in the next few days to incorporate the corresponding changes as stated in the discussion here: https://github.com/cogment/cogment-verse/pull/176