search

facebookresearch / torchbeast

A PyTorch Platform for Distributed RL
Apache License 2.0
734 stars 113 forks source link

Multi-agent RL extension? #18

Closed MachengShen closed 3 years ago

MachengShen commented 3 years ago

I'm wondering if it would be straightforward to extend to the multi-agent case without breaking the multi-node multi-gpu training capability?

heiner commented 3 years ago

Hey MachengShen, thanks for your interest in this project!

For multi-agent RL, there's a number of ways one can do this. The simplest would be to multiplex the environment on an "agent" dimension where both the observation and the action space gets this additional agent dimension. This could be done easily with e.g. a wrapper around the environment that does this.

There's also more complex ways of doing this which allow for more general setups. One example would be the "Rela" agent in e.g. https://github.com/facebookresearch/hanabi_SAD.

MachengShen commented 3 years ago

Hey, heiner, thanks for the quick reply! will try it out