search

reiniscimurs / DRL-robot-navigation

Deep Reinforcement Learning for mobile robot navigation in ROS Gazebo simulator. Using Twin Delayed Deep Deterministic Policy Gradient (TD3) neural network, a robot learns to navigate to a random goal point in a simulated environment while avoiding obstacles.
MIT License
571 stars 119 forks source link

if it supports multi-agent navigation? #57

Closed SHITIANYU-hue closed 11 months ago

SHITIANYU-hue commented 1 year ago

May i know if this one has gym environment https://github.com/reiniscimurs/DRL-robot-navigation/tree/main/catkin_ws/src/multi_robot_scenario?

reiniscimurs commented 1 year ago

Hi,

Sorry, I am not quite sure what is the question? For the project, there is a Gazebo environment. You will have to be more specific what you mean by "gym" environment in this context.

SHITIANYU-hue commented 1 year ago

Sorry for the confusion, i want to know if it could support multi-robot training?

SHITIANYU-hue commented 1 year ago

Similar to this kind of scenario: https://www.youtube.com/watch?v=BBLrnqkzpAg

reiniscimurs commented 1 year ago

Not by default. The neural network supports batch inputs so you can have multiple outputs in a single pass and produce outputs for multiple robots. But the gazebo environment spawns a single robot and returns the state for a single robot. This repo is not for multi-robot training in this implementation, so without changes this repo is for a single robot training only.

However, it is possible to spawn multiple robots and rewrite the environment file to support mult-robot training. This does require quite some changes to the code and is not straight forward. After that, to make the neural network learn what you want would be entirely up to your network design.

SHITIANYU-hue commented 1 year ago

Hi, thanks for your reply! Could you please provide some suggestions for how to rewrite the environment file to support mult-robot training?

reiniscimurs commented 1 year ago

You would have to spawn multiple robots in the launch file: https://github.com/reiniscimurs/DRL-robot-navigation/blob/main/TD3/assets/multi_robot_scenario.launch For this I would suggest to use turtlebot branch as turtlebot launch files are easier to work with.

Then you would have to rewrite the environment file to read topics for all the robots you are spawning: https://github.com/reiniscimurs/DRL-robot-navigation/blob/main/TD3/velodyne_env.py#L81-L131 All the parts that take single robot inputs would need to be made to calculate for each of your robots. Ideally, you would refactor the code to use some sort of robot classes and pass them dynamically through the environment as currently it is hardcoded for a single robot. It is not all that difficult but it is a bit finicky and fixing any ROS issues that might arise are a pain in the butt.