search

reiniscimurs / DRL-robot-navigation

Deep Reinforcement Learning for mobile robot navigation in ROS Gazebo simulator. Using Twin Delayed Deep Deterministic Policy Gradient (TD3) neural network, a robot learns to navigate to a random goal point in a simulated environment while avoiding obstacles.
MIT License
488 stars 98 forks source link

Training does not converge #49

Closed PinkViolet closed 1 year ago

PinkViolet commented 1 year ago

Hi,

Thank for your great jobs and sorry for bothering you. I trained the code for a day but it does not converge, please help me to solve the problem. Thanks in advance.

Here are my system setups:

Problems I have: I trained the policy for a day, but the rewards do not increase and the robot tends to turn right instead of moving towards to the goal.

.............................................. Average Reward over 10 Evaluation Episodes, Epoch 62: -92.909436, 0.800000 ..............................................

Meanwhile, below errors occurred periodically :

Exception in thread /r1/odom: Traceback (most recent call last): File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_pubsub.py", line 185, in robust_connect_subscriber conn.receive_loop(receive_cb) File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 846, in receive_loop self.close() File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 858, in close self.socket.close() AttributeError: 'NoneType' object has no attribute 'close'

I found the above errors mentioned in #8 and I suppose these problems are solved in the main branch based on the discussion in this issue. In addition, i'm aware of that the Melodic branch is deprecated. However, I need to stick with ROS melodic for my project. And I notice main branch code can be run in melodic setup mentioned in #48. Should I give it a try?

reiniscimurs commented 1 year ago

One thing to check would be the rqt tree, see if there is any issue there. And then go from there. Did you try retraining again or multiple times with different seeds?

If it is possible to run the Noetic branch in Melodic, that would probably be the way to go because I have made some updates to Noetic codebase (unrelated to the ROS version) that are not present in the older branch. It would make sense to try it at least.

PinkViolet commented 1 year ago

Hi,

Sorry for getting back late. Thanks for helping out.

I successfully solved the problem by training a model using Noetic branch and loaded the checkpoints trained on Noetic branch to Melodic branch.