How can I reproduce experimental results?

[hyoonsoo]

Hello, I am very impressed with the CoPO project. Thank you for sharing a great paper and code. I wanted to see the trained multi-agent, so I visualized it using the weight stored in copo_code/copo/best_checkpoint/ and copo_code/vis.py. file. (without any modifications) However, unlike the paper, I was able to render agents with lower performance(lower succeess rate). How should I modify the code to see the higher performance of agents like your paper? I look forward to your reply. Thank you.

[pengzhenghao]

This is due to the changes of the environments. MetaDrive multiagent benchmark became much harder!

I will update models and exp results soon!

Best regards!

Peng Zhenghao (彭正皓)

在 2022年9月8日，11:32，IMO @.***> 写道：

 Closed #25 as completed.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

[pengzhenghao]

Hi Yoonsoo,

Finally, I finished benchmarking the results of various MARL algorithms in MetaDrive MARL environments. Please kindly refer to this page:

https://github.com/metadriverse/metadrive-benchmark/tree/main/MARL

And I also upload latest trained models so you can run it to visualize the behaviors! This time I don’t find any performance discrepancy in the latest models (which proved that the performance discrepancy is due to the update of environment)!

https://github.com/decisionforce/CoPO#visualization

Thanks!

[hyoonsoo]

I've been waiting for a new update!! Thank you for your kind reply.

[pengzhenghao]

As I updated last month, the result is in https://github.com/metadriverse/metadrive-benchmark/tree/main/MARL

Do you have any question?

[XilunZhangRobo]

Hi Zhenghao, I trained the intersection using torch copo (train_copo.py) and tried to evaluate the performance using copo_code/new_vis.py. However, it gives unpickled = pickle.loads(data) TypeError: an integer is required (got type bytes). I used the file that was stored in TEST/CoPOTraininger_Multi.../checkpoint_000440/algorithm_state.pkl. Did I use the wrong pkl file?

[pengzhenghao]

Hi Zhenghao, I trained the intersection using torch copo (train_copo.py) and tried to evaluate the performance using copo_code/new_vis.py. However, it gives unpickled = pickle.loads(data) TypeError: an integer is required (got type bytes). I used the file that was stored in TEST/CoPOTraininger_Multi.../checkpoint_000440/algorithm_state.pkl. Did I use the wrong pkl file?

Hi @XilunZhangRobo

Let's discuss this in a new issue.