How to transfer the GRF_MARL environment and use other MARL algorithms on it?

[GP413413]

Thank you for your surprising work! I have successfully run the project and I have some questions. The algorithms in GRF_MARL, including MAPPO HAPPO MAT , are implemented with the model. I want to add some new MARL algorithms, or only extract the environment contents of GRF_MARL for tarining. What should I do and where need to make some modifications? I hope for your advice and thank for your help :)

[YanSong97]

Hi there, I have added instructions for contribution in the README file. We also have an example of DDPG algorithm here.

To transfer our version of GRF environment, you can copy the directory envs/gr_football and import it in a similar way as in light_malib/registry/registration.py.

We welcome all kinds of contributions. Please let me know if you encounter any issues :)

[GP413413]

Hello, thank you for your reply. As for transferring GRF environment, directly copying the directory light_malib/envs/gr_football may not work. I am not quite sure about how to import it in a similar way as in light_malib/registry/registration.py. Please give me some more detailed procedures. Again thanks for your help :)

[YanSong97]

Hi, can you try from light_malib.envs.gr_football.env import GRFootballEnv and init as env = GRFootballEnv(id, seed, cfg) ?

[GP413413]

Thank you for your advice, and I will have a try. Another question is , I want to add other algorithms into https://github.com/jidiai/GRF_MARL/tree/main/light_malib/algorithm or https://github.com/jidiai/GRF_MARL/tree/main/light_malib/model/gr_football, but the algorithm does not have Actor and Critic class. What should I do to fit the algorithm into GRF_MARL program, or if it is necessary to rewrite the algorithm into Actor-Critic structure and add the yaml in https://github.com/jidiai/GRF_MARL/tree/main/expr_configs/cooperative_MARL_benchmark/full_game/11_vs_11_hard. Thank you for your help :)

[YanSong97]

The policy class in policy.py has actor and critic initialized (see line). And the details of actor/critic network are shown in /model (see line).

[GP413413]

Thank you for your suggestions. I have met some difficulties in transferring your version of GRF environment, the directory envs/gr_football includes the main contents of modified GRF environment but I have to tranfer the whole light-malib due to the associations between env.py with other files like light_malib/utils. I wonder if your team has wrapped your version of GRF environment into a universal python lib like gfootball in https://github.com/google-research/football, and whether the GRF_MARL environment supports communication-based MARL algorithms. Thank you for your help :)

[YanSong97]

The environment we provided is more like an external wrapper over the origin GRF scenario and no we don't have a universal lib. I believe you can add help functions in /utils if necessary since they are mainly for logging purposes. Also what communication-based MARL algorithms are you referring to?

[GP413413]

Thank you for your explanation. Actually, I am working on multi-agent communication, and recently focus on the open source of https://github.com/charleshsc/CommFormer and its comparative baselines included in https://github.com/CORE-Robotics-Lab/MAGIC, mainly the latter. This MAGIC repo is adapted from IC3Net, which also added GRF environment but do not support 11 vs 11 hard game scenario.
I want to tranfer your version of GRF_MARL environment into MAGIC for testing, but I have met some difficulties since their frameworks differ a lot. I am still trying to find some ways to solve it. Any suggestions from you are valuable :) Thank you for your help.

[GP413413]

Hello. I wonder if all players in GRF_MARL environment have global observation or only have partial observability. Thank you for your help.

[YanSong97]

By default, they have access to global states. You can also set up partial observability in feature encoder if necessary.

[GP413413]

Thank you for your help!