search

reiniscimurs / DRL-robot-navigation

Deep Reinforcement Learning for mobile robot navigation in ROS Gazebo simulator. Using Twin Delayed Deep Deterministic Policy Gradient (TD3) neural network, a robot learns to navigate to a random goal point in a simulated environment while avoiding obstacles.
MIT License
488 stars 98 forks source link

Test DRL in turtlebot #41

Closed blackjacket996 closed 1 year ago

blackjacket996 commented 1 year ago

Hey bro. I trained the model and saw it worked well in the Rviz simulation, but when I put the model into my Turtle bot, and make the Velodyne data, Odom, and goal position as the input, it cannot work well. So I wanna ask for some advice about what should I pay more attention to when I use DRL in real robot platform.

Sincerely!

reiniscimurs commented 1 year ago

Hi,

So you are running a real turtlebot with the neural network? Did I get that right? In that case you have to make sure that your real world implementation is as close to the simulation as possible. Did you train in simulation with a turtlebot model? And what sensors are you using as inputs?

blackjacket996 commented 1 year ago

I use Velodyne VLP16 as inputs and I did not simulate with a turtlebot, oh! I think I should make sure the velodyne lidar is in the same position in the robot as that in simulation world. Is that right?

reiniscimurs commented 1 year ago

So, first thing you should check is how the data is ordered on the real VLP16. In the simulation we set the range of -90 to 90 degree FoV and it works out of the box. For the real VLP16 (if I remember correctly) you can not go to -90 degrees when setting it up. It can only go for 0 to 180 degrees. Also a thing to remember is that VLP16 the 0 degree is facing directly away from the cable. So you can either limit the VLP in 0 to 180 degree range and place it on the robot sideways, or take the full 0 to 360 degree range. In the latter case, you would have to filter out the unused 180 degrees to obtain 2 ranges of 0 to 90 degrees and 270 to 360 degrees. Then you must place all the values in a list, but the 270 to 360 range needs to come before the 0 to 90 range. That would sort of get the -90 to 90 degree range of the simulation.

Another thing is that the laser is most likely at a different height in your implementation than in simulation. Since in simulation we filter out the floor based on the height of where the sensor is located, this will cause some problems. Luckily, all you need to do here is find the right height of your real sensor and filter out the floor in the incoming data.

Third issue is that if you are using a different model of robot, the assumed (learned) kinematics are not going to be the same. In simulation the robot model has certain height and width, that determines how it will act. Giving it learned velocities, the outcome is learned. But on a different model, if you give it the same velocities, it might actually have larger or smaller turning rates. This of course will cause weird behavior. For this, you either should re-train the neural network with the proper model, or find some right coefficients for the network output. For instance input to turtlebot = rotational velocity * X where X is some coefficient that you can find experimentally.

blackjacket996 commented 1 year ago

OH ! I see！ Thank you so much, I will correct my experiment.