Add support for python 3.10 (and remove pybullet)

[pseudo-rnd-thoughts]

Description

CleanRL has been limited to <3.10 due to PyBullet and IssacGym. Due to updating to Gymnasium which includes Mujoco-v4 elevating the need for PyBullet as Mujoco is now freely available. This PR provides support for python 3.10 and removes PyBullet.

This should have no impact on the performance of algorithms.

Types of changes

• [ ] Bug fix
• [x] New feature
• [ ] New algorithm
• [ ] Documentation

Checklist:

• [x] I've read the CONTRIBUTION guide (required).
• [x] I have ensured pre-commit run --all-files passes (required).
• [x] I have updated the tests accordingly (if applicable).
• [x] I have updated the documentation and previewed the changes via mkdocs serve.
  • [ ] I have explained note-worthy implementation details.
  • [ ] I have explained the logged metrics.
  • [ ] I have added links to the original paper and related papers.

If you need to run benchmark experiments for a performance-impacting changes:

• [ ] I have contacted @vwxyzjn to obtain access to the openrlbenchmark W&B team.
• [ ] I have used the benchmark utility to submit the tracked experiments to the openrlbenchmark/cleanrl W&B project, optionally with --capture-video.
• [ ] I have performed RLops with python -m openrlbenchmark.rlops.
  • For new feature or bug fix:
    • [ ] I have used the RLops utility to understand the performance impact of the changes and confirmed there is no regression.
  • For new algorithm:
    • [ ] I have created a table comparing my results against those from reputable sources (i.e., the original paper or other reference implementation).
  • [ ] I have added the learning curves generated by the python -m openrlbenchmark.rlops utility to the documentation.
  • [ ] I have added links to the tracked experiments in W&B, generated by python -m openrlbenchmark.rlops ....your_args... --report, to the documentation.

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
cleanrl	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 13, 2023 9:44am

[pseudo-rnd-thoughts]

@vwxyzjn I have tested installed the PettingZoo group locally with no issue and checking the pypi of Tinyscaler (https://pypi.org/project/tinyscaler/), I can't see any issues. Any ideas?

[vwxyzjn]

@vwxyzjn I have tested installed the PettingZoo group locally with no issue and checking the pypi of Tinyscaler (https://pypi.org/project/tinyscaler/), I can't see any issues. Any ideas?

Probably because tinyscalar does have Mac wheels https://pypi.org/project/tinyscaler/1.2.5/#files. If this is a problem, feel free to remove the pettingzoo Mac CI.