nsidn98 / InforMARL

Code for our paper: Scalable Multi-Agent Reinforcement Learning through Intelligent Information Aggregation
https://nsidn98.github.io/InforMARL/
MIT License
72 stars 22 forks source link

Edge formation #10

Closed Michael-Elrod-dev closed 6 months ago

Michael-Elrod-dev commented 6 months ago

Hello, thank you for your help and patience while I learn from this project. I had one more question for you.

While training and evaluating I see that edges are not only created between agents and other entities but the entities also share edges (e.g. goal to goal). I wanted to ask about the benefits of doing this if there are any. I imagine that if we were to increase the number of agents and therefore the number of goal positions and obstacles that the number of edges created could potentially grow exponentially.

In your tests during the creation of this project did you recognize any benefits to creating these edges between non-agent entities or does this feature not impact performance in a meaningful way and it is just a visual representation?

nsidn98 commented 6 months ago

Hey, no worries at all! Happy to answer any questions you might have!

Yes, that is a good catch! You are right, we do not want to have all the edges between non-agent entities in the graphs. We filter out those edges from the graph in our experiments. I am not able to find that part in the code snippet in the repo right now but as far as I can remember it was in this function: https://github.com/nsidn98/InforMARL/blob/304e905d05b34d9bf06046eb7e03904b97a14231/onpolicy/algorithms/utils/gnn.py#L324 I think I missed to push that change here and I am not able to find that part in my local machine. If you are willing to submit a PR with that filter added on which takes out all the non-agent to non-agent entity edges out from the adjacency matrix, it would be really helpful!

Thanks!