search

proroklab / VectorizedMultiAgentSimulator

VMAS is a vectorized differentiable simulator designed for efficient Multi-Agent Reinforcement Learning benchmarking. It is comprised of a vectorized 2D physics engine written in PyTorch and a set of challenging multi-robot scenarios. Additional scenarios can be implemented through a simple and modular interface.
https://vmas.readthedocs.io
GNU General Public License v3.0
346 stars 68 forks source link

Training Hyperparameters for VMAS Reproduction #147

Open Square596 opened 3 weeks ago

Square596 commented 3 weeks ago

Hello,

Thank you for this helpful repository! I’m trying to reproduce the results of experiments from one of your papers, VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learning, using TorchRL. Specifically, I’m focusing on reproducing the results shown in Figure 4, but I haven’t been able to find certain training hyperparameters (e.g., number of epochs per iteration, batch size, optimizer, learning rate, GAE lambda) in the paper or code.

Could you provide any details on these?

Thank you for your help!

matteobettini commented 2 weeks ago

Hello!

Thanks for openeing this.

So, those experiment were run using RLLib with a setup similar to the one here (https://github.com/proroklab/HetGPPO/blob/main/train/train_give_way.py).

I think the hyperparameters should be those, the one i linked is give_way but other scenarios should be similar.

Since torchrl is different from rllib i would first try to reproduce in the original rllib setup and then try to match that with the colsest one in rllib

The H1 section of this paper https://matteobettini.com/publication/torchrl-a-data-driven-decision-making-library-for-pytorch/TorchRL-A-data-driven-decision-making-library-for-PyTorch.pdf might help with the mapping of hyperparameters from rllib to torch rl