search

reiniscimurs / DRL-robot-navigation

Deep Reinforcement Learning for mobile robot navigation in ROS Gazebo simulator. Using Twin Delayed Deep Deterministic Policy Gradient (TD3) neural network, a robot learns to navigate to a random goal point in a simulated environment while avoiding obstacles.
MIT License
488 stars 98 forks source link

Unconnected tf trees #48

Closed JasonHysy closed 1 year ago

JasonHysy commented 1 year ago

Hello!

Sorry for taking your time. I’m new to ROS and I came across some issues with Rivz display. When the global fixed frame is set to Odom, the marker array shows but other criteria fails to load, when I switch the global fixed frame to base_link, the other criteria works but the marker arrays fails.

I checked and it is possibly caused by the unconnected tf trees between baselink and Odom. The tf tree in my scenario looked like this: frames.pdf Does the tf tree looked the same on your side or I’m loading the robot wrong? Also if I want to connect these two tf trees, which file I could go to so that I can make modifications?

Thank you in advance!

reiniscimurs commented 1 year ago

Hi,

Please let me know your system setup and installation steps (ROS, Ubuntu versions, installation steps). Also, please print out your console log.

JasonHysy commented 1 year ago

Hello!

My system setup was ROS melodic, with Ubuntu 18.04 I followed the exact installation steps in README.md, and it went smoothly.

The training process I had the same issue, however it appears that the robot is trained properly. Looking from Gazebo it was able to navigate to the goal point with an increasing average reward. It might be just some issue with Rviz.

With that said, here is my console log running test_velodyne_td3.py, since I am running it right now. Please let me know if you need any other information!

$ python3 test_velodyne_td3.py Roscore launched! Unable to register with master node [http://localhost:11311]: master may not be running yet. Will keep trying. ... logging to /home/jason/.ros/log/b9287b10-9b3a-11ed-9353-f0b61e7da785/roslaunch-jason-MS-7D30-4320.log Checking log directory for disk usage. This may take a while. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://localhost:45155/ ros_comm version 1.14.13

SUMMARY

PARAMETERS

NODES

auto-starting new master process[master]: started with pid [4356] ROS_MASTER_URI=http://localhost:11311/

setting /run_id to b9287b10-9b3a-11ed-9353-f0b61e7da785 process[rosout-1]: started with pid [4390] started core service [/rosout] Gazebo launched! ... logging to /home/jason/.ros/log/b9287b10-9b3a-11ed-9353-f0b61e7da785/roslaunch-jason-MS-7D30-4399.log Checking log directory for disk usage. This may take a while. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is <1GB.

Deprecated: xacro tag 'property' w/o 'xacro:' xml namespace prefix (will be forbidden in Noetic) when processing file: /home/jason/DRL-robot-navigation/catkin_ws/src/multi_robot_scenario/xacro/p3dx/inertia_tensors.xacro included from: /home/jason/DRL-robot-navigation/catkin_ws/src/multi_robot_scenario/xacro/p3dx/pioneer3dx_body.xacro included from: /home/jason/DRL-robot-navigation/catkin_ws/src/multi_robot_scenario/xacro/p3dx/pioneer3dx.xacro Use the following command to fix incorrect tag usage: find . -iname "*.xacro" | xargs sed -i 's#<([/]\?)(if|unless|include|arg|property|macro|insert_block)#<\1xacro:\2#g'

started roslaunch server http://localhost:32955/

SUMMARY

PARAMETERS

NODES / gazebo (gazebo_ros/gzserver) joint_state_publisher (joint_state_publisher/joint_state_publisher) robot_state_publisher (robot_state_publisher/robot_state_publisher) rviz (rviz/rviz) urdf_spawner (gazebo_ros/spawn_model)

ROS_MASTER_URI=http://localhost:11311/

process[gazebo-1]: started with pid [4465] process[urdf_spawner-2]: started with pid [4469] process[robot_state_publisher-3]: started with pid [4471] process[joint_state_publisher-4]: started with pid [4472] process[rviz-5]: started with pid [4473] [ INFO] [1674491204.576754689]: Finished loading Gazebo ROS API Plugin. [ INFO] [1674491204.577379200]: waitForService: Service [/gazebo/set_physics_properties] has not been advertised, waiting... [INFO] [1674491204.751053, 0.000000]: Loading model XML from ros parameter robot_description [INFO] [1674491204.756289, 0.000000]: Waiting for service /gazebo/spawn_urdf_model [ INFO] [1674491204.943357397]: waitForService: Service [/gazebo/set_physics_properties] is now available. [ INFO] [1674491204.948302227]: Physics dynamic reconfigure ready. [INFO] [1674491205.058189, 0.000000]: Calling service /gazebo/spawn_urdf_model [ INFO] [1674491205.294805870, 0.201000000]: Camera Plugin: Using the 'robotNamespace' param: '/' [ INFO] [1674491205.295870647, 0.201000000]: Camera Plugin (ns = /) , set to "" [ INFO] [1674491205.299445810, 0.201000000]: Camera Plugin: The 'robotNamespace' param was empty [ INFO] [1674491205.300440173, 0.201000000]: Camera Plugin (ns = r1) , set to "r1" [ INFO] [1674491205.362810918, 0.201000000]: Laser Plugin: The 'robotNamespace' param was empty [ INFO] [1674491205.362849321, 0.201000000]: Starting Laser Plugin (ns = r1) [ INFO] [1674491205.363304670, 0.201000000]: Laser Plugin (ns = r1) , set to "r1" [INFO] [1674491205.903600, 0.201000]: Spawn status: SpawnModel: Successfully spawned entity [ INFO] [1674491205.906843237, 0.201000000]: Velodyne laser plugin missing , defaults to no clipping [ INFO] [1674491205.907539781, 0.201000000]: Velodyne laser plugin ready, 16 lasers [ INFO] [1674491206.007369485, 0.201000000]: Starting plugin DiffDrive(ns = r1/) [ INFO] [1674491206.007415744, 0.201000000]: DiffDrive(ns = r1/): = Debug [ INFO] [1674491206.007616425, 0.201000000]: DiffDrive(ns = r1/): = r1 [DEBUG] [1674491206.007638252, 0.201000000]: DiffDrive(ns = r1/): = cmd_vel [DEBUG] [1674491206.007643762, 0.201000000]: DiffDrive(ns = r1/): = odom [DEBUG] [1674491206.007648809, 0.201000000]: DiffDrive(ns = r1/): = odom [DEBUG] [1674491206.007653524, 0.201000000]: DiffDrive(ns = r1/): = base_link [DEBUG] [1674491206.007669459, 0.201000000]: DiffDrive(ns = r1/): = false [ WARN] [1674491206.007674490, 0.201000000]: DiffDrive(ns = r1/): missing default is true [DEBUG] [1674491206.007681544, 0.201000000]: DiffDrive(ns = r1/): = true [DEBUG] [1674491206.007706789, 0.201000000]: DiffDrive(ns = r1/): = 0.29999999999999999 [DEBUG] [1674491206.007723647, 0.201000000]: DiffDrive(ns = r1/): = 0.17999999999999999 [DEBUG] [1674491206.007730821, 0.201000000]: DiffDrive(ns = r1/): = 1.8 [DEBUG] [1674491206.007736523, 0.201000000]: DiffDrive(ns = r1/): = 20 [DEBUG] [1674491206.007741395, 0.201000000]: DiffDrive(ns = r1/): = 50 [DEBUG] [1674491206.007763367, 0.201000000]: DiffDrive(ns = r1/): = world := 1 [DEBUG] [1674491206.007774713, 0.201000000]: DiffDrive(ns = r1/): = left_hub_joint [DEBUG] [1674491206.007783153, 0.201000000]: DiffDrive(ns = r1/): = right_hub_joint [ WARN] [1674491206.007801793, 0.201000000]: GazeboRosDiffDrive Plugin (ns = ) missing , defaults to 1 [ INFO] [1674491206.008049498, 0.201000000]: DiffDrive(ns = r1/): Advertise joint_states [ INFO] [1674491206.008296666, 0.201000000]: DiffDrive(ns = r1/): Try to subscribe to cmd_vel [ INFO] [1674491206.008947613, 0.201000000]: DiffDrive(ns = r1/): Subscribe to cmd_vel [ INFO] [1674491206.009065044, 0.201000000]: DiffDrive(ns = r1/): Advertise odom on odom [ INFO] [1674491206.011388413, 0.201000000]: GazeboRosJointStatePublisher is going to publish joint: chassis_swivel_joint [ INFO] [1674491206.011400256, 0.201000000]: GazeboRosJointStatePublisher is going to publish joint: swivel_wheel_joint [ INFO] [1674491206.011406171, 0.201000000]: GazeboRosJointStatePublisher is going to publish joint: left_hub_joint [ INFO] [1674491206.011411320, 0.201000000]: GazeboRosJointStatePublisher is going to publish joint: right_hub_joint [ INFO] [1674491206.011416333, 0.201000000]: Starting GazeboRosJointStatePublisher Plugin (ns = r1/)!, parent name: r1 [DEBUG] [1674491206.014904451, 0.205000000]: Trying to publish message of type [sensor_msgs/Image/060021388200f6f0f447d0fcd9c64743] on a publisher with type [sensor_msgs/Image/060021388200f6f0f447d0fcd9c64743] [DEBUG] [1674491206.015030639, 0.205000000]: Trying to publish message of type [sensor_msgs/CameraInfo/c9a58c1b0b154e0e6da7578cb991d214] on a publisher with type [sensor_msgs/CameraInfo/c9a58c1b0b154e0e6da7578cb991d214] [DEBUG] [1674491206.021303348, 0.211000000]: Trying to publish message of type [sensor_msgs/LaserScan/90c7ef2dc6895d81024acba2ac42f369] on a publisher with type [sensor_msgs/LaserScan/90c7ef2dc6895d81024acba2ac42f369] [DEBUG] [1674491206.032846752, 0.216000000]: Trying to publish message of type [sensor_msgs/PointCloud2/1158d486dd51d683ce2f1be655c3c181] on a publisher with type [sensor_msgs/PointCloud2/1158d486dd51d683ce2f1be655c3c181] [DEBUG] [1674491206.038860676, 0.222000000]: Trying to publish message of type [nav_msgs/Odometry/cd5e73d190d741a2f92e81eda573aca7] on a publisher with type [nav_msgs/Odometry/cd5e73d190d741a2f92e81eda573aca7] [DEBUG] [1674491206.038901289, 0.222000000]: Trying to publish message of type [sensor_msgs/JointState/3066dcd76a6cfaef579bd0f34173e9fd] on a publisher with type [sensor_msgs/JointState/3066dcd76a6cfaef579bd0f34173e9fd] [urdf_spawner-2] process has finished cleanly log file: /home/jason/.ros/log/b9287b10-9b3a-11ed-9353-f0b61e7da785/urdf_spawner-2*.log QObject::connect: Cannot queue arguments of type 'QVector' (Make sure 'QVector' is registered using qRegisterMetaType().) QObject::connect: Cannot queue arguments of type 'QVector' (Make sure 'QVector' is registered using qRegisterMetaType().)

Screenshot from 2023-01-23 17-50-28 Screenshot from 2023-01-23 17-50-19

reiniscimurs commented 1 year ago

Hi,

ROS melodic version has a separate branch in this repository. It should still work, however, consider it depreciated. If you installed the main branch, that supports only Noetic version and has issues running in Melodic.

You could however, create a static link between r1/base_link and base_link. You can check in this file to see how to create static transform publisher: https://github.com/reiniscimurs/DRL-robot-navigation/blob/9a4d92ca41e370617baa4451f0a784f8ca5239ce/TD3/assets/multi_robot_scenario.launch

JasonHysy commented 1 year ago

Thank you so much for this! I will look more into it and really appreciate your help!

JasonHysy commented 1 year ago

Hello! I have successfully made everything render in Rviz, with however just one issue. now that I connected all of the trees, linked below: frames.pdf

the odometries and marker array is fixed on map, while the laserscan is fixed on the robot. Is there any way I could fix this?

Screenshot from 2023-01-23 19-25-57

reiniscimurs commented 1 year ago

AFAIK, the base_link should not be directed towards odom. It should be the other way around.

Judging by your first qrt_tree that you showed, your direction of links should be: r1/odom -> r1/base_link -> base_link -> the rest of the tree

So you would only need to create a link between r1/base_link -> base_link

I would suggest trying that

JasonHysy commented 1 year ago

Yes it worked!

For reference, here is the tf tree working for me: frames.pdf

by adding these codes to the TD3/assets/multi_robot_scenario.launch

<node pkg="tf" type="static_transform_publisher" name="r1baselink_to_baselink" args="0.0 -0.0 0.0 0.0 0.0 0.0 r1/base_link base_link 200" /> <node pkg="tf" type="static_transform_publisher" name="odom_to_r1odom" args="0.0 -0.0 0.0 0.0 0.0 0.0 odom r1/odom 200" /> <node pkg="tf" type="static_transform_publisher" name="laser_to_r1laser" args="0.0 -0.0 0.0 0.0 0.0 0.0 front_laser r1/front_laser 200" /> <node pkg="tf" type="static_transform_publisher" name="camera_to_r1frontcamera" args="0.0 -0.0 0.0 0.0 0.0 0.0 front_camera r1/front_camera 200" />

JasonHysy commented 1 year ago

sorry one more question. My GPU and CPU are at around 10~20% when training this model. Is there any parameters I could change to improve the hardware utilization?

reiniscimurs commented 1 year ago

Great that it worked out!

The network here isn't particularly heavy, so it shouldn't use a lot of resources. But you could increase the batch size, that would increase the amount of data over which calculations are made in a single pass. This could also increase the speed of convergence but it is not guaranteed.

JasonHysy commented 1 year ago

yes I tried that the convergence seemed faster. Thank you again for all of your help and this amazing project!