Remove Need for Setuptools and Pip Downgrades

[Zach-Attach]

Is your feature request related to a problem? Please describe. As it currently stands, users are required to run pip install setuptools==65.5.0 pip==21 prior to installing. This is a solution, but is fragile and may lead to dependency conflicts in the future. It also makes installation of the package more difficult for users and may force them to be unable to run it with their existing code.

Describe the solution you'd like According to this comment, the solution is to change gym to gymnasium

• All references to gym would be replaced by gymnasium
• Any incompatibilities should be adjusted as specified here
• The following dependencies would need to be updated:
  • gym==0.21 ->gymnasium
  • stable-baselines3[extra]==1.8.0 -> stable-baselines[extra]==2
  • sb3-contrib==1.8.0 -> sb3_contrib==2
  • git+https://github.com/ir413/mvp -> fork of mvp that uses gymnasium such as (https://github.com/Zach-Attach/mvp)
  • NOTE: parts of mvp uses isaacgym which uses gym==0.21, so this would need to be accounted for
  • mlagents==1.0.0 -> mlagents with gymnasium, although the current dependency is gym>=0.21 so gym==0.27 might work

It might be possible to simply use gym==0.27 instead, but this is unconfirmed and just speculative.

Describe alternatives you've considered

• Continue to use gym==0.21 until mlagents updates its dependency to gymnasium
• Update to gym==0.27 if it solves the issue, this would still require dependency updates although the scope of this is not certain. At the very least, mvp would need to be forked and changed to be compatible with gym==0.27

[desaibhargav]

Thanks for the details on this @Zach-Attach, the only clean solution I see is if mlagents bumps support from gym to gymnasium which currently does not look like a priority, see here [https://github.com/Unity-Technologies/ml-agents/issues/5995]

I haven't tested with gym==0.27 yet, please let me know if everything runs successfully for you with that version.

[desaibhargav]

"Nothing planned for now. MLA is in maintenance mode for now. Minor bug fixes, dependency updates, minor features. We are, however, accepting PRs, so if anyone from the community would like to make this upgrade and submit a PR, that would be amazing." is the latest update from devs at mlagents,

We might consider sending a PR to them for this at some point, so keeping this issue open.