search

reiniscimurs / DRL-robot-navigation

Deep Reinforcement Learning for mobile robot navigation in ROS Gazebo simulator. Using Twin Delayed Deep Deterministic Policy Gradient (TD3) neural network, a robot learns to navigate to a random goal point in a simulated environment while avoiding obstacles.
MIT License
569 stars 119 forks source link

how to speed up the training process? #110

Closed des-zhong closed 5 months ago

des-zhong commented 7 months ago

Hi, really great projects. But i have some questions about the speeding up process of the training. In your tutorial "https://medium.com/@reinis_86651/deep-reinforcement-learning-in-mobile-robot-navigation-tutorial-part5-some-extra-stuff-b744852345ac", you mentioned about changing the factor in TD3.world to say 2000 which will speed up the simulation by 2 times. But I've noticed that in velodyne_env.py file, the step function utilized time.sleep(TIME_DELTA) function to excute the given action for TIME_DELTA time. I don't think changing the factor will affect the total time for an episode which may still remain TIME_DELTA*episode_length How to solve this? Thank you for your time

reiniscimurs commented 7 months ago

Hi, yes you would have to reduce the TIME_DELTA accordingly by factor of the increased the simulation time. Simulation time is separate from TIME_DELTA (which uses computer time). If you do not do this, you would still get sort of a speedup as your single step would be propagated for a longer time span, however that is not the intended behavior.