[Tutorials] MARL tutorial is not really MARL

[smarianimore]

• [ ] I have marked all applicable categories:
  • [ ] exception-raising bug
  • [ ] RL algorithm bug
  • [X] documentation request (i.e. "X is missing from the documentation.")
  • [ ] new feature request
• [X] I have visited the source website
• [X] I have searched through the issue tracker for duplicates
• [X] I have mentioned version numbers, operating system and environment, where applicable:

Premise

Hi everybody, congratulations for the good work on this interesting library :)

Issue description

The tutorial available here has a misleading title and description IMHO, as the code provided does not really perform multi-agent RL: only 1 agent is learning, the other is a random policy or a pre-trained one.

Solution proposal

Easiest IMHO would be to replace the random policy agent with another instance of the other DQN agent (sharing no params, independent learning setting).

Software stack info

tianshou.version='0.5.1', gym.version='0.29.1', torch.version='2.1.0+cu121', numpy.version='1.26.0', sys.version='3.11.3 (main, Apr 17 2023, 12:18:57) [GCC 11.3.0]', sys.platform='linux'

[MischaPanch]

The whole MARL stuff is a bit deprioritized now, but if you feel like it's an easy fix in the documentation, I'd be happy to review a PR :)

[pietrofusco]

Hi everybody, congratulations for this interesting library!

I have two questions: 1) Can a custom PettingZoo Parallel or an AEC environment be used? If yes, is it possible to get an example/tutorial? 2) When discussing MARL, are you referring to an environment with more than two agents?

Regards,