Open ardian-selmonaj opened 1 month ago
Hello, we are very sorry that we have not considered the operation of saving and loading a specific agent model for MARL. At present, saving and loading models in XuanCe are used for the holistic MARL models. We will consider fixing this issue in the future version of XuanCe. Thanks very much for your questions.
This would be very helpful! I like your overall framework for training and evaluating RL agents and the variety of algorithms, but accessing specific MARL policies for inference is indeed an important feature.
Thank you for your advice; we will consider supporting this function very soon.
Additionally, can you also consider adding a feature to train selected agents with a shared policy? e.g. ag1
and ag2
use the same pi1
but ag3
uses pi2
and so on. This would allow to train the same type of agents in a heterogeneous setting with the same policies.
It's a good suggestion for more general scenarios, implementing this feature in the current version of XuanCe sounds not easy. We may need to rebuild the policy module of XuanCe to determine whether or not some agents use shared policy.
Ray RLlib supports both these features, however their variety of algorithms is limited. Anyhow, I would be very thankful if you could provide these features soon. I assume many others would benefit from it.
In a multi-agent setting, when training e.g.
MAPPO_Agents()
, then callingMAPPO_Agents.save_model(model_name='model.pth')
and finally loading the modelMAPPO_Agents.load_model(path)
, how can I extract specific policies out of the model? e.g. when 3 agents were trained, "agent_1", "agent_2" and "agent_3", I would like to have the prediction only of a specific agent.I didn't find out how to exactly do that and would be very thankful for any help.