search

TonghanWang / RODE

Codes accompanying the paper "RODE: Learning Roles to Decompose Multi-Agent Tasks (ICLR 2021, https://arxiv.org/abs/2010.01523). RODE is a scalable role-based multi-agent learning method which effectively discovers roles based on joint action space decomposition according to action effects, establishing a new state of the art on the StarCraft multi-agent benchmark.
Apache License 2.0
69 stars 20 forks source link

Can't the RODE algorithm run in parallel? #1

Closed jk96491 closed 3 years ago

jk96491 commented 4 years ago

Hello

I want to run RODE in parallel

However, looking at the code, the parallel execution side is not able to respond.

Is it impossible to run in parallel?

TonghanWang commented 4 years ago

Hi, we are sorry that we only prepare the episode_runner. And we recommend this runner for the SMAC benchmark.

The reason is that, typically, on SMAC, parallel environments will not improve the performance. For example, if you achieve 100% win rate using the episode_runner at 1M steps, you will get 100% at 8M if you use 8 parallel environments. This is the same for many popular algorithms, like QMIX, QPLEX, etc.

We will update the code to support parallel_runner. We will let you know as soon as we finish it. Sorry again for the inconvenience.

jk96491 commented 4 years ago

TonghanWang //

Thanks for the reply.!!! Even if it takes a lot of time, it should be done by episode.

TonghanWang commented 4 years ago

You are welcome! I'd like to clarify one more thing that I didn't make it clear.

By the examples (1M steps vs. 8M steps) in the previous response, I mean it takes roughly the same time (wall clock) for both episode_runner and parallel runner to achieve the same win rate. If you use episode_runner, you only need to run 2M steps, while using 8 parallel environments, you roughly need 16M steps.

I guess this is also the reason why other popular algorithms like QMIX use episode_runner too.