any implement for car following with ddpg?

[laoyouf]

Hi, All. Nice work! I would like to ask if your sumo-rl library can implement multi-agent vehicle control? For example, I am trying to make a scene: 1km straight single-lane for car following . There are two kinds of vehicles on the road, one is controlled by IDM and the other is controlled by DDPG algorithm. How can I use your environment to implement such an example?

[m-kro]

Any sumo reinforcement learning library you may encounter is made by third parties (which may have their own Github repo or website). This Github repo is only about the original SUMO suite and its components.

[laoyouf]

Any sumo reinforcement learning library you may encounter is made by third parties (which may have their own Github repo or website). This Github repo is only about the original SUMO suite and its components.

Thank you for your answer, perhaps I should not initiate an issue here, I will close this issue

[laoyouf]

I'm sorry, I have reopened this issue. May I ask if you are aware that other third-party libraries are doing this implentment? Or have anyone else done similar things in previous issues? I don't seem to have found any relevant issues

[m-kro]

A quick online search gives me this publication: Berahman, M.; Rostami-Shahrbabaki, M.; Bogenberger, K. Multi-Task Vehicle Platoon Control: A Deep Deterministic Policy Gradient Approach. Future Transp. 2022, 2, 1028-1046. https://doi.org/10.3390/futuretransp2040057 If you don't limit your search to car-following, there are other works referring to DDPG. E.g. this one.

[laoyouf]

A quick online search gives me this publication: Berahman, M.; Rostami-Shahrbabaki, M.; Bogenberger, K. Multi-Task Vehicle Platoon Control: A Deep Deterministic Policy Gradient Approach. Future Transp. 2022, 2, 1028-1046. https://doi.org/10.3390/futuretransp2040057 If you don't limit your search to car-following, there are other works referring to DDPG. E.g. this one.

great help! thanks for your kindness, I will check the info.

[robertalms]

You may have a look at this: stable-baselines3. It's a well maintained and documented library that allows you to create custom environments (incl. SUMO) and also deploy a variety of algorithms.

[laoyouf]

You may have a look at this: stable-baselines3. It's a well maintained and documented library that allows you to create custom environments (incl. SUMO) and also deploy a variety of algorithms.

thanks, I do know the SB3 library, but there is no similar implentment for car following based on DDPG with SUMO, I am wondering if any implenttment like this is ready for a beginner to get start

[robertalms]

I am not aware of any out-of-the-box solution with a custom SUMO environment for such a specific task. This is a quite advanced RL task you're trying to tackle. I doubt you'll find something that is just ready-to-use, and I would recommend you to try your best starting from scratch with your idea, anyways. The paper mentioned before is already quite on point and a good ressource. You may also have a look at flow, but they stopped maintaining their repo. Anyways, good luck with your research. Let us know if you got something to work...

[laoyouf]

I am not aware of any out-of-the-box solution with a custom SUMO environment for such a specific task. This is a quite advanced RL task you're trying to tackle. I doubt you'll find something that is just ready-to-use, and I would recommend you to try your best starting from scratch with your idea, anyways. The paper mentioned before is already quite on point and a good ressource. You may also have a look at flow, but they stopped maintaining their repo. Anyways, good luck with your research. Let us know if you got something to work...

Yes, thanks to your kindness, I am now trying to set up such an RL environment myself