search

DongChen06 / MARL_CAVs

MARL for Autonomous Vehicles
240 stars 45 forks source link

About MAPPO with Safety Supervisor and Action Mask #46

Open gyccccccccc opened 3 months ago

gyccccccccc commented 3 months ago

Hi! Thank you for your great job! Recently I want to add the action mask module(just change the actor network in model_common.py as the network in model.py ) and Safety Supervisor into MAPPO, but I find the performance of it is worse than MAPPO itself. I couldn't find out the reason. So I really want to get your help, Could you tell me the safety supervisor can work in the mappo( I mean the performance of mappo with safety supervisor can be better than mappo) , or this module can only work in the maa2c? (ps: the first image is the mappo with safety supervisor , the second is only mappo itself)

图片1

图片2

DongChen06 commented 3 months ago

Hi thanks for your interest. The safety supervisor can also be adapted to the MAPPO algorithm with few code changes. I am highly confident that it will improve the performance. The action masking may also improve the MAPPO. Intuitively, it works as well...

gyccccccccc commented 3 months ago

Thank you for your kindly reply, I just find that I've made some mistakes! By the way, Could I ask another question? I wanna generate the vehicles trajectories in mappo.py, and use it in safety supervisor in abstract.py. Is that OK?

gyccccccccc commented 3 months ago

Hi thanks for your interest. The safety supervisor can also be adapted to the MAPPO algorithm with few code changes. I am highly confident that it will improve the performance. The action masking may also improve the MAPPO. Intuitively, it works as well...

Wishing for your kind reply