search

SafeRL-Lab / Safe-MARL-in-Autonomous-Driving

Safe Multi-Agent Reinforcement Learning to Make decisions in Autonomous Driving.
Apache License 2.0
24 stars 5 forks source link

Baseline Algorithm Code #1

Closed lycheeelf closed 1 month ago

lycheeelf commented 1 month ago

three other algorithms: Bi-AC, MACPO, and MAPPO-LI’ve been exploring the CSQ and CS-MADDPG algorithms in your repository, and I’m impressed by their performance. However, I noticed that the baseline algorithms (Bi-AC, MACPO, and MAPPO-L) are not open-sourced.

As a fellow researcher interested in multi-agent reinforcement learning (MARL), I believe having access to the baseline code would greatly enhance our understanding and facilitate further comparisons. Would it be possible to release the code for these baselines?

Thank you for your valuable work, and I appreciate your consideration.

xiaoBOSS97 commented 1 month ago

Hi, thank you for your valuable comment. The baseline algorithm Bi-AC is already included in this repository. You can switch to Bi-AC algorithm by changing the argument 'arg.version' in main_bilevel.py. But the Bi-AC algorithm in my rep is modified from its original version, which can be found in https://github.com/laonahongchen/Bilevel-Optimization-in-Coordination-Game. The other two algorithms(MACPO and MAPPO-L) can be found from https://github.com/chauncygu/Multi-Agent-Constrained-Policy-Optimisation. I think all of them are open-sourced.

lycheeelf commented 1 month ago

Thanks for your reply. Is the args.version == "bilevel" parameter, as referenced in the unconstrained Stackelberg MADDPG, related to your modified Bi-AC algorithm? I noticed that this parameter is activated when args.action_type == "continuous". In the merging scenario where the action type is set to discrete, how can I obtain the output of the Bi-AC algorithm similar to the file ./merge_env_result/exp1_bilevel? Thanks a lot for your kind assistance!

xiaoBOSS97 commented 1 month ago

My mistake. That parameter is used for continuous action type. Since the basic discrete action framework for Bi-AC and CSQ are similar, you can run Bi-AC simply by changing the parameter "cost_threshold" in "config.json" from experiment folder to "inf" or a really large number. Because Bi-AC is the unconstrained version of CSQ.

lycheeelf commented 1 month ago

Got it, thanks! All the best to you.