Unable to apply Q-learning baselines on envs with non-homogeneous agents

[zez2001]

Your q-learning baselines assume all agents have same obs and action dim, however in MPE_Single_Tag there are two different obs sizes.How can i apply qmix.py on MPE_Single_Tag?

[mttga]

Hi @zez2001,

The Q-learning algorithms do support environments with different observation/action spaces. These are unified by the CTRolloutManager.

You can easily run the Q-learning algorithms on MPE_simple_tag_v3 (I assume you're referring to this scenario?) with an environment configuration like this (let's say it is baselines/QLearning/config/env/mpe_simple_tag.yaml):

"ENV_NAME": "MPE_simple_tag_v3"
"ENV_KWARGS": {}

and by running:

python baselines/QLearning/qmix.py +alg=qmix_mpe +env=mpe_simple_tag

However, I'm not sure about running QMIX on this environment. Simple tag is a cooperative-competitive environment, and QMIX is designed only for cooperative tasks. I would focus on iql instead.

Let me know if this helps.

[zez2001]

Thank you for your answer,

I check your CTRolloutManager,and it seems that you pad the observation vectors to the maximum length.I wonder whether i can use this on MPE_simple_tag_v3,cause they have different obs size.If i just pad the vector with values =0,maybe i will get the inappropriate params when i apply them into q-network?

[mttga]

This is a standard practice in marl. Closing this because I don't see any codebase issues.