search

reiniscimurs / DRL-robot-navigation

Deep Reinforcement Learning for mobile robot navigation in ROS Gazebo simulator. Using Twin Delayed Deep Deterministic Policy Gradient (TD3) neural network, a robot learns to navigate to a random goal point in a simulated environment while avoiding obstacles.
MIT License
635 stars 126 forks source link

Changing robot to an center articulated four wheeled robot #117

Closed zonared closed 7 months ago

zonared commented 9 months ago

Hi there Reinis, thankyou so much for writing such a complete deep learning ROS system, it was pretty fun to watch the pioneer3dx robot learn how to navigate, then come back a day later and it was happily finding its goal, so cool.

So, being inspired by that, I thought I would try to load my robot into the sim, which for the most part I have (see below), but I need to tweak a few things in your DRL code but I'm not sure quite how. Firstly my robot interprets angular velocity as steering angle (like TEB local planner cmd_angle_instead_rotvel, will this effect things much? Also steering is limited +-0.5rad. Also my robot can drive backwards and I cant quite find the right place allow this.

How would all these changes effect the actor and critic weights and so on? Also I note that min_laser feeds into the rewards as well, but my robot is a rectangle, so a radius value might not work that well, maybe? Again, thanks heaps.

artmule_sim

reiniscimurs commented 9 months ago

Hi,

  1. You should be able to use steering angle in principle. The TD3 code does not change, you should just change the execution command in the env file. The TD3 architecture is not tied to how the action is executed and it does not know what action values actually mean.
  2. The actions are capped to ranges here: https://github.com/reiniscimurs/DRL-robot-navigation/blob/main/TD3/train_velodyne_td3.py#L344-L345
  3. For backwards driving you probably need to use different sensor FOV. The current implementation is hardcoded to 180 degree view in the env file. But you can update this as you want.
  4. For actor/critic weights short answer is - I don't know. I have not tried to train such robots and do not have insights there. Feel free to try out different parameters. I think at least in theory it all should work.
  5. You could try updating min_laser based reward function. But if you are still using collision based on a single min_laser value, I think it would be unnecessary. What this reward enforces is for the robot to keep larger distance from obstacles. And even in your case it quite easily be a circle.
zonared commented 2 weeks ago

Hi there Reinis, sorry for the later reply to a closed topic. I'm not even sure I'm meant to post to a closed topic. But I'll keep it short just in case I'm not meant to post here. I have been trying random things with just about every part of the TD3 code since posting my OG question to make it work with my robot geometry and none of my changes result in convergence. I wont dive into the details until I hear from you just in case I'm meant to start a new issue before hand. Thanks again, cheers.

reiniscimurs commented 2 weeks ago

Hi,

I do not think I have anything to add here that I did not mention in the comment above. Feel free to open a new issue and provide all the information, but please use the template when submitting the issue.