Documentation for multi-agent needs fixing

[destin-v]

• [x] I have marked all applicable categories:
  • [ ] exception-raising bug
  • [ ] RL algorithm bug
  • [x] documentation request (i.e. "X is missing from the documentation.")
  • [ ] new feature request
  • [ ] design request (i.e. "X should be changed to Y.")
• [x] I have visited the source website
• [x] I have searched through the issue tracker for duplicates
• [x] I have mentioned version numbers, operating system and environment, where applicable:

Description

The current documentation on the tic tac toe example found in the documentation does not work. This was tested on tianshou==1.0.0 and pettingzoo==1.24.3 which are the latest as of Jun 2024.

🐛 Errors

Some issues are related to the fact that the class names have changed or the arguments have changed. But even after fixing the names, it still throws errors.

self.train_collector.reset_stat()
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'int' object has no attribute 'reset_stat'

🐛 PettingZoo Examples Broken

The Tianshou examples on PettingZoo also seem to be broken here, here, and here. These are likely related and should be fixed.

[MischaPanch]

Yes, some examples are quite outdated - they're the only part of the code that is still untested in CI. Thanks for reporting!

The plan for the next weeks is to introduce an script that runs all examples and reproduces the benchmark results that are now reported in the docs. Then all problematic scripts will be adjusted. Fortunately, with @maxhuettenrauch's last contributions we got rliable-style evaluations in tianshou, now we just need to adjust the example scripts.

However, I want to already mention that all MARL topics are deprioritized atm, so some examples might actually be dropped or marked as incomplete