search

reiniscimurs / DRL-robot-navigation

Deep Reinforcement Learning for mobile robot navigation in ROS Gazebo simulator. Using Twin Delayed Deep Deterministic Policy Gradient (TD3) neural network, a robot learns to navigate to a random goal point in a simulated environment while avoiding obstacles.
MIT License
486 stars 97 forks source link

Why does the robot's perform poor during testing compare to training phase? #136

Open am-amani opened 2 months ago

am-amani commented 2 months ago

Dear @reiniscimurs

I have been using Turtlebot Waffel Pi and Burger Autorace with 2D lidar. During training, the robot performed very well and reached the target goal. However, I have noticed a strange behavior during the testing phase. The robot sometimes gets stuck in the following behaviors:

Please check the following two videos for more clarity: https://github.com/reiniscimurs/DRL-robot-navigation/assets/33548412/4c6ea1a5-bc39-4847-ac93-71ad2b461159 https://github.com/reiniscimurs/DRL-robot-navigation/assets/33548412/e8941948-0a44-4de3-8ecc-b32e258f59f7

I am using the simple laser scan on the turtles (default one) with the following descriptions:

<gazebo reference="base_scan">
    <material>Gazebo/FlatBlack</material>
    <sensor type="ray" name="lds_lfcd_sensor">
      <pose>0 0 0 0 0 0</pose>
      <visualize>$(arg laser_visual)</visualize>
      <update_rate>100</update_rate>
      <ray>
        <scan>
          <horizontal>
        <samples>360</samples>
            <resolution>1</resolution>
            <min_angle>-1.5708</min_angle>
            <max_angle>1.5708</max_angle>les>
          </horizontal>
        </scan>
        <range>
          <min>0.19</min>
          <max>12.0</max>
          <resolution>0.01</resolution>
        </range>
        <noise>
          <type>gaussian</type>
          <mean>0.0</mean>
          <stddev>0.01</stddev>
        </noise>
      </ray>
      <plugin name="gazebo_ros_lds_lfcd_controller" filename="libgazebo_ros_laser.so">
        <topicName>scan</topicName>
        <frameName>base_scan</frameName>
      </plugin>
    </sensor>
  </gazebo>

Also, I noticed that the laser scan callback could be faster when calculating the min range scan. The laser scan suddenly jumps when the robot moves, and an obstacle is in front of it. For example, the value of minimum scan data is 0.88, 0.83, 0.80, 0.45! Is this because of the GAZEBO or laser scan description?

reiniscimurs commented 2 months ago

Most likely your model has encountered a local optimum. Since there is no history information in the states, the robot gets stuck in a cycle where left action is followed by right action that leads again back to the left action. See more on the discussion about that here: https://github.com/reiniscimurs/DRL-robot-navigation/issues/89

For the sudden laser jumps, that would depend entirely on the situation at hand, if it is a problem or not. Without more information it is difficult to say what is happening there