search

flow-project / flow

Computational framework for reinforcement learning in traffic control
MIT License
1.05k stars 370 forks source link

Add an implementation of "Learning Multi-agent Communication with Backpropagation" to Flow #707

Open eugenevinitsky opened 4 years ago

eugenevinitsky commented 4 years ago

It would be interesting to port a few basic communication environments/training procedures to Flow. In particular, a popular communications baselines is "Learning Multi-agent Communication with Backpropagation" (https://arxiv.org/abs/1605.07736). As a request for research, we'd appreciate it if someone added the training procedure from this paper to our codebase. For open source reproductions of the training process, you can look here: https://github.com/KornbergFresnel/CommNet

To be clearer what would be great is: (1) Take any of our existing environments, add vehicle to vehicle communication to them as part of the action space. (2) Add a script to train in the new environment and use the Commnet as the model.

jennac3 commented 4 years ago

Hi Eugene, I'm interested in contributing to this feature request. Would I be able to be an assignee for this feature?

eugenevinitsky commented 4 years ago

Oh my god yes that'd be amazing. I've assigned you.

eugenevinitsky commented 4 years ago

Please reach out if you get stuck or need help.

jennac3 commented 4 years ago

Hi Eugene, I've noticed that we currently use ray.rllib for the reinforcement learning algorithm. Do you suggest implementing CommNet from scratch to replace the existing RL algorithms supported by ray.rllib?

eugenevinitsky commented 4 years ago

Hi Jenna, So in this case of the CommNet, there isn't an algorithmic change, it's just a model. So you would just need to implement the model; the actual construction of a policy gradient would just be handled by rllib.

ramondalmau commented 4 years ago

Dear all, was it finally implemented? :)

eugenevinitsky commented 4 years ago

Nope! Still would love someone to do this.

ramondalmau commented 4 years ago

Excellent, I will probably work on that since I definitely need to add some communication between my agents. I was also thinking on implementing the model proposed in "Connectivity-driven Communication in Multi-agent Reinforcement Learning through Diffusion Processes on Graphs". I will keep you in-the-loop. Thanks.

eugenevinitsky commented 4 years ago

Please! We'd love any implementation you have, I've been meaning to play around with communication in these environments for YEARS.

rohitmodee commented 1 year ago

Any updates on this!