Multi Agent in Intersection for RL algorithm

[SExpert12]

Hi all, I want to apply reinforcement learning using multi agent, specifically algorithms are PPO, TRPO, DDPG and A2C. I don't understand how to write Carla environment for these algorithm. Is any documentation or implementation is available?

Thanks for your time.

[PatrickPromitzer]

Hi, I am not sure how you want to train your algorithms, but you could download the Carla server and look at the examples of the Carla API. Carla API examples: https://github.com/carla-simulator/carla/tree/dev/PythonAPI/examples

Carla Server: https://github.com/carla-simulator/carla/releases

[SExpert12]

Yes I know about PythonAPI. What I wanted is to train many cars based with reinforcement learning algorithm for decision making and for the same I needed help. Thanks for reply.

[PatrickPromitzer]

You can start with this example: https://github.com/carla-simulator/carla/blob/dev/PythonAPI/examples/sensor_synchronization.py It will keep the Client and server synchronized, will spawn sensors and

For controlling a vehicle, you can look into this example https://github.com/carla-simulator/carla/blob/dev/PythonAPI/examples/vehicle_physics.py

After you have a vehicle which can be controlled, sensor data, and simulation data, you need to get the data to the reinforcement learning algorithm. I am not sure how you want to create the connection between the algorithm and the simulation, but you may need "Queue" if it is in a sub process.

I guess it is probably like this: reinforcement learning algorithm

• get data (Queue?)
• process data
• send output (Queue?)
• repeat

Carla API

• get data from the Simulation
• send data to the reinforcement learning algorithm (Queue?)
• get data from the reinforcement learning algorithm (Queue?)
• send data to the Simulation (update steering, break, ...)
• world.tick()
• repeat

[SExpert12]

Hi, thanks for reply. I need to create an environment in CARLA for reinforcement learning. That is the main problem in RL algorithm.

[PatrickPromitzer]

Hi, if I understood it right, you mean the standard maps are not enough for you. (for example, Town10 is a map) If you want to create a custom map, you need the source version. https://carla.readthedocs.io/en/0.9.15/build_carla/

With the source version you can create a custom map, add new objects, add new Vehicles, and more.

The next version of Carla will most likely be in Unreal Engine 5 (Currently Unreal Engine 4.26 is used).

Is this the information you are looking for?

[SExpert12]

Hi, Can I have your mail id? So I can communicate in a better way.

[PatrickPromitzer]

Hi, I want to avoid switching to mail, because I just want to help out the community with answering some issues. I am not in the Carla team and just answer some issues to help out.

I don't have the time to answer issues, work on my project, and make a one-to-one deep help talk. It would be another story if I have to do it for work (or if I get paid for it), but I don't think that is the case in this situation.