Comparison algorithms

[Yu-zx]

If I want to verify different algorithms for the same scenario, how should I write the code to verify the algorithm to support different algorithms?

[nsidn98]

You can find the scripts used for the baselines here: https://github.com/nsidn98/InforMARL/tree/main/scripts. You might need to change the filepaths appropriately.

If you want to test your custom algorithm on it, could you please give me some more information on it so that I could refer you to relevant files to modify.

[Yu-zx]

If I want to test the file multiagent/custom_scenarios/navigation_graph.py, how should I write the related train.py and run scripts? I think there may be some problems in this regard. Can you give a detailed explanation?

[Yu-zx]

For this comparison algorithm, do I need to ignore the relationships between graph modeling and keep the other MDPs unchanged and build the scenario in one step? Or do I need to keep this scenario file? Which one will work as expected?

[nsidn98]

If you want to use the navigation_graph.py for testing a different algorithm, you will need to appropriately extract the observations, graph-observations (adjacency matrix, node features, edge features), etc. (refer here).

Then you will have to modify the trainer (refer here). Here self.trainer is the GR_MAPPO class (refer here).

So you will need to modify the MAPPO algorithm's loss update function according to your algorithm's loss function.

I hope this answers your question:)

[Yu-zx]

Maybe you misunderstood what I meant. Let me ask whether other algorithms can be used to verify it. What I mean is that when using the algorithm below the baseline, do you understand what I mean? Or do we need to reconstruct an MDP without graphical observations?

[nsidn98]

Sorry, I still do not understand what do you mean by "algorithm below the baseline". Can you please elaborate a bit?

[Yu-zx]

I roughly understand it. What I want to ask is how should I run those algorithm comparison scripts at the same time? Can those sh files be run directly?

[nsidn98]

These scripts should ideally work. Let me know if they don't.

[Yu-zx]

What I mean is, if I want to compare this algorithm with some new algorithms that have come out, how should I do it? Would it be convenient to give some details?

[nsidn98]

By algorithm, do you mean the underlying RL algorithm or the architecture? If you could let me know the modifications in the new algorithm, I could let you know which files are supposed to be changed.

[nsidn98]

Closing because of inactivity. Please reopen if the issue still persists.

[Yu-zx]

I would like to ask if I want to use distributed reinforcement learning algorithms such as ippo for this scene file, how should I add it or what is the more appropriate way to train it?