search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.12k stars 4.15k forks source link

Feature request: upgrade to gymnasium/update pettingzoo version #5890

Closed elliottower closed 1 year ago

elliottower commented 1 year ago

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Hi, would it be possible add compatibility with Gymnasium and newer versions of PettingZoo?

Gymnasium and PettingZoo are compatible with current RL training libraries (rllib, tianshou and CleanRL have already migrated, and stable-baselines3 will soon) as well as other tools such as Comet and WandB.

For information about upgrading and compatibility, see migration guide and gym compatibility. The main difference is the API has switched to returning truncated and terminated, rather than done, in order to give more information and mitigate edge case issues.

We would like to list this package in the third party environments lists (PettingZoo, Gymnasium), but are removing all libraries which do not plan to migrate to Gymnasium/recent versions of PettingZoo.

Describe the solution you'd like A clear and concise description of what you want to happen.

This package would be updated to use gymnasium rather than gym, and use at least PettingZoo v1.20 (uses termination/truncation and gymnnasium rather than gym).

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

elliottower commented 1 year ago

Hi, just bumping this as I would love to be able to use gymnasium/pettingzoo with unity ml-agents. I would be happy to contribute any help which is needed, just wanted to see if this is on the radar and how difficult it would be.

We are cutting a new pettingzoo release shortly with some important fixes (no major breaking changes) and I was thinking after that is out it would be great to try and update this project to be compatible. Gymnasium had a new release V28 recently and I would be happy to contribute to make that compatible as well.

miguelalonsojr commented 1 year ago

Thanks for the request. We're planning these updates for our next release.

elliottower commented 1 year ago

Thanks for the request. We're planning these updates for our next release.

Great to hear, thanks! We are planning a major pettingzoo release either this coming Tuesday or the following Tuesday. The only breaking change is removing the “return_info” argument for reset(), to match gymnasium and always return an info dict. We also fixed a number of bugs, and don’t intend to do any other major releases for a while, so it would be great if ml-agents could be compatible with it.

I would be happy to help double check that things are fully compatible if you want help.

github-actions[bot] commented 1 year ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.