No map setting in Rviz

[acarrou]

I'm running this application with ros melodic. The camera works fine, however, it doesn't save the points and I'm not seeing the setting for map in TF.

[MartyG-RealSense]

Hi @acarrou Intel have published a guide to setting up SLAM in the RealSense ROS wrapper with the D435i camera model.

https://github.com/IntelRealSense/realsense-ros/wiki/SLAM-with-D435i

At the bottom of that guide, it states that the point cloud can be saved with this command:

rosrun pcl_ros pointcloud_to_pcd input:=/rtabmap/cloud_map

This is presumably dependent on rtabmap_ros being installed as a prerequisite, as the top of the guide suggests.

[acarrou]

Hi @acarrou Intel have published a guide to setting up SLAM in the RealSense ROS wrapper with the D435i camera model.

https://github.com/IntelRealSense/realsense-ros/wiki/SLAM-with-D435i

At the bottom of that guide, it states that the point cloud can be saved with this command:

rosrun pcl_ros pointcloud_to_pcd input:=/rtabmap/cloud_map

This is presumably dependent on rtabmap_ros being installed as a prerequisite, as the top of the guide suggests.

Sorry for the late reply. I was using this guide but I am not able to map out the area and I believe it's because I don't have one of the settings called map in TF shown in this image.

[MartyG-RealSense]

In this particular tutorial, Fixed Frame in RViz is set to map. Do you have that setting defined in your Fixed Frame please?

https://raw.githubusercontent.com/wiki/intel-ros/realsense/videos/os_tracking_process.gif

[acarrou]

In this particular tutorial, Fixed Frame in RViz is set to map. Do you have that setting defined in your Fixed Frame please?

https://raw.githubusercontent.com/wiki/intel-ros/realsense/videos/os_tracking_process.gif

I do have it defined as map, but it has errors in some of the other parts. I'm not sure what's going on because I installed everything that is needed.

[MartyG-RealSense]

Comparing your Frames topic list to the one in the guide, it looks as though you are missing the IMU and infrared topics from your list. Since that guide was first published, the RealSense ROS wrapper was changed in wrapper 2.2.15 onwards to make the IMU and infrared topics disabled by default.

You can enable the IMU and infrared topics in the roslaunch statement by adding the commands below to the end of it:

enable_gyro:=true enable_accel:=true enable_infra:=true

[acarrou]

Yes, I have heard them changing this, and thank you. After running roslaunch realsense2_camera opensource_tracking.launch enable_gyro:=true enable_accel:=true enable_infra:=true I have the same issue although I do have this error.

[rtabmap/rgbd_odometry-5] process has died [pid 23301, exit code 127, cmd /home/adrien/catkin_ws/devel/lib/rtabmap_ros/rgbd_odometry --delete_db_on_start rgb/image:=/camera/color/image_raw depth/image:=/camera/aligned_depth_to_color/image_raw rgb/camera_info:=/camera/color/camera_info rgbd_image:=rgbd_image_relay odom:=odom imu:=/imu/data __name:=rgbd_odometry __log:=/home/adrien/.ros/log/9af9c786-3203-11eb-bfe5-14abc5292ac5/rtabmap-rgbd_odometry-5.log].
log file: /home/adrien/.ros/log/9af9c786-3203-11eb-bfe5-14abc5292ac5/rtabmap-rgbd_odometry-5*.log
[ERROR] [1606627666.058909381]: Client [/rviz] wants topic /rtabmap/info to have datatype/md5sum [rtabmap_ros/Info/1348655e70ba2e47fa77d7c83e936d33], but our version has [rtabmap_ros/Info/dd8620be5f7bd87dac1670b11dc3540d]. Dropping connection.

Not sure why it's being so difficult but thank you for all the help!

[acarrou]

Comparing your Frames topic list to the one in the guide, it looks as though you are missing the IMU and infrared topics from your list. Since that guide was first published, the RealSense ROS wrapper was changed in wrapper 2.2.15 onwards to make the IMU and infrared topics disabled by default.

You can enable the IMU and infrared topics in the roslaunch statement by adding the commands below to the end of it:

enable_gyro:=true enable_accel:=true enable_infra:=true

Interestingly enough, I tried doing this on my desktop and it works. Could this be because it requires USB 3.0 and a good gpu?

[MartyG-RealSense]

Regarding this error:

Client [/rviz] wants topic /rtabmap/info to have datatype/md5sum [rtabmap_ros/Info/1348655e70ba2e47fa77d7c83e936d33], but our version has [rtabmap_ros/Info/dd8620be5f7bd87dac1670b11dc3540d]. Dropping connection.

I have not seen this RTABMAP related error in RealSense ROS before, though a small number of RTABMAP users have encountered it. One person who had it found that it was caused by having two different versions of RTABMAP installed on their computer. If you have a similar situation then it may help to account for it working with one computer but not the other.

http://official-rtab-map-forum.67519.x6.nabble.com/RTABMAP-on-a-robot-with-client-side-visualization-td3843.html

[Honza0297]

Hello! Unfortunately, I have run into similar issue as acarrou. I have followed the guide to set up SLAM (https://github.com/IntelRealSense/realsense-ros/wiki/SLAM-with-D435i), but unfortunately, it did not work. Camera works well, but I am unable to map the area. Below, there is a start of the log and a screenshot from rviz with my Frames topic list. Could somebody tell what can be wrong? Thanks in advance!

Rviz screenshot:

Start of the output:

$ roslaunch realsense2_camera opensource_tracking.launch enable_accel:=true enable_gyro:=true enable_infra:=true
... logging to /home/trilobot/.ros/log/efcb5a72-325d-11eb-9d31-c3da6a7f365a/roslaunch-trilo6-40545.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://trilo6:46349/

SUMMARY
========

CLEAR PARAMETERS
 * /ukf_se/

PARAMETERS
 * /ImuFilter/_publish_tf: False
 * /ImuFilter/_world_frame: enu
 * /ImuFilter/use_mag: False
 * /camera/realsense2_camera/accel_fps: 250
 * /camera/realsense2_camera/accel_frame_id: camera_accel_frame
 * /camera/realsense2_camera/accel_optical_frame_id: camera_accel_opti...
 * /camera/realsense2_camera/align_depth: True
 * /camera/realsense2_camera/aligned_depth_to_color_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_fisheye1_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_fisheye2_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_fisheye_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_infra1_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_infra2_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/allow_no_texture_points: False
 * /camera/realsense2_camera/base_frame_id: camera_link
 * /camera/realsense2_camera/calib_odom_file: 
 * /camera/realsense2_camera/clip_distance: -2.0
 * /camera/realsense2_camera/color_fps: 30
 * /camera/realsense2_camera/color_frame_id: camera_color_frame
 * /camera/realsense2_camera/color_height: 480
 * /camera/realsense2_camera/color_optical_frame_id: camera_color_opti...
 * /camera/realsense2_camera/color_width: 640
 * /camera/realsense2_camera/depth_fps: 30
 * /camera/realsense2_camera/depth_frame_id: camera_depth_frame
 * /camera/realsense2_camera/depth_height: 480
 * /camera/realsense2_camera/depth_optical_frame_id: camera_depth_opti...
 * /camera/realsense2_camera/depth_width: 640
 * /camera/realsense2_camera/device_type: 
 * /camera/realsense2_camera/enable_accel: True
 * /camera/realsense2_camera/enable_color: True
 * /camera/realsense2_camera/enable_depth: True
 * /camera/realsense2_camera/enable_fisheye1: False
 * /camera/realsense2_camera/enable_fisheye2: False
 * /camera/realsense2_camera/enable_fisheye: False
 * /camera/realsense2_camera/enable_gyro: True
 * /camera/realsense2_camera/enable_infra1: False
 * /camera/realsense2_camera/enable_infra2: False
 * /camera/realsense2_camera/enable_infra: True
 * /camera/realsense2_camera/enable_pointcloud: False
 * /camera/realsense2_camera/enable_pose: False
 * /camera/realsense2_camera/enable_sync: False
 * /camera/realsense2_camera/filters: 
 * /camera/realsense2_camera/fisheye1_frame_id: camera_fisheye1_f...
 * /camera/realsense2_camera/fisheye1_optical_frame_id: camera_fisheye1_o...
 * /camera/realsense2_camera/fisheye2_frame_id: camera_fisheye2_f...
 * /camera/realsense2_camera/fisheye2_optical_frame_id: camera_fisheye2_o...
 * /camera/realsense2_camera/fisheye_fps: 30
 * /camera/realsense2_camera/fisheye_frame_id: camera_fisheye_frame
 * /camera/realsense2_camera/fisheye_height: 480
 * /camera/realsense2_camera/fisheye_optical_frame_id: camera_fisheye_op...
 * /camera/realsense2_camera/fisheye_width: 640
 * /camera/realsense2_camera/gyro_fps: 400
 * /camera/realsense2_camera/gyro_frame_id: camera_gyro_frame
 * /camera/realsense2_camera/gyro_optical_frame_id: camera_gyro_optic...
 * /camera/realsense2_camera/imu_optical_frame_id: camera_imu_optica...
 * /camera/realsense2_camera/infra1_frame_id: camera_infra1_frame
 * /camera/realsense2_camera/infra1_optical_frame_id: camera_infra1_opt...
 * /camera/realsense2_camera/infra2_frame_id: camera_infra2_frame
 * /camera/realsense2_camera/infra2_optical_frame_id: camera_infra2_opt...
 * /camera/realsense2_camera/infra_fps: 30
 * /camera/realsense2_camera/infra_height: 480
 * /camera/realsense2_camera/infra_rgb: False
 * /camera/realsense2_camera/infra_width: 640
 * /camera/realsense2_camera/initial_reset: False
 * /camera/realsense2_camera/json_file_path: 
 * /camera/realsense2_camera/linear_accel_cov: 1.0
 * /camera/realsense2_camera/odom_frame_id: camera_odom_frame
 * /camera/realsense2_camera/pointcloud_texture_index: 0
 * /camera/realsense2_camera/pointcloud_texture_stream: RS2_STREAM_COLOR
 * /camera/realsense2_camera/pose_frame_id: camera_pose_frame
 * /camera/realsense2_camera/pose_optical_frame_id: camera_pose_optic...
 * /camera/realsense2_camera/publish_odom_tf: True
 * /camera/realsense2_camera/publish_tf: True
 * /camera/realsense2_camera/rosbag_filename: 
 * /camera/realsense2_camera/serial_no: 
 * /camera/realsense2_camera/tf_publish_rate: 0.0
 * /camera/realsense2_camera/topic_odom_in: odom_in
 * /camera/realsense2_camera/unite_imu_method: linear_interpolation
 * /camera/realsense2_camera/usb_port_id: 
 * /points_xyzrgb/approx_sync: True
 * /points_xyzrgb/decimation: 4.0
 * /points_xyzrgb/voxel_size: 0.0
 * /rosdistro: noetic
 * /rosversion: 1.15.8
 * /rtabmap/rgbd_odometry/approx_sync: True
 * /rtabmap/rgbd_odometry/config_path: 
 * /rtabmap/rgbd_odometry/expected_update_rate: 0.0
 * /rtabmap/rgbd_odometry/frame_id: camera_link
 * /rtabmap/rgbd_odometry/ground_truth_base_frame_id: 
 * /rtabmap/rgbd_odometry/ground_truth_frame_id: 
 * /rtabmap/rgbd_odometry/guess_frame_id: 
 * /rtabmap/rgbd_odometry/guess_min_rotation: 0.0
 * /rtabmap/rgbd_odometry/guess_min_translation: 0.0
 * /rtabmap/rgbd_odometry/max_update_rate: 0.0
 * /rtabmap/rgbd_odometry/odom_frame_id: odom
 * /rtabmap/rgbd_odometry/publish_tf: True
 * /rtabmap/rgbd_odometry/queue_size: 10
 * /rtabmap/rgbd_odometry/subscribe_rgbd: False
 * /rtabmap/rgbd_odometry/wait_for_transform_duration: 0.2
 * /rtabmap/rgbd_odometry/wait_imu_to_init: False
 * /rtabmap/rtabmap/Mem/IncrementalMemory: true
 * /rtabmap/rtabmap/Mem/InitWMWithAllNodes: false
 * /rtabmap/rtabmap/approx_sync: True
 * /rtabmap/rtabmap/config_path: 
 * /rtabmap/rtabmap/database_path: ~/.ros/rtabmap.db
 * /rtabmap/rtabmap/frame_id: camera_link
 * /rtabmap/rtabmap/ground_truth_base_frame_id: 
 * /rtabmap/rtabmap/ground_truth_frame_id: 
 * /rtabmap/rtabmap/landmark_angular_variance: 9999.0
 * /rtabmap/rtabmap/landmark_linear_variance: 0.0001
 * /rtabmap/rtabmap/map_frame_id: map
 * /rtabmap/rtabmap/odom_frame_id: 
 * /rtabmap/rtabmap/odom_sensor_sync: False
 * /rtabmap/rtabmap/odom_tf_angular_variance: 1.0
 * /rtabmap/rtabmap/odom_tf_linear_variance: 1.0
 * /rtabmap/rtabmap/publish_tf: True
 * /rtabmap/rtabmap/queue_size: 10
 * /rtabmap/rtabmap/scan_cloud_max_points: 0
 * /rtabmap/rtabmap/subscribe_depth: True
 * /rtabmap/rtabmap/subscribe_odom_info: True
 * /rtabmap/rtabmap/subscribe_rgb: True
 * /rtabmap/rtabmap/subscribe_rgbd: False
 * /rtabmap/rtabmap/subscribe_scan: False
 * /rtabmap/rtabmap/subscribe_scan_cloud: False
 * /rtabmap/rtabmap/subscribe_scan_descriptor: False
 * /rtabmap/rtabmap/subscribe_stereo: False
 * /rtabmap/rtabmap/subscribe_user_data: False
 * /rtabmap/rtabmap/wait_for_transform_duration: 0.2
 * /ukf_se/acceleration_gains: [0.8, 0.0, 0.0, 0...
 * /ukf_se/acceleration_limits: [1.3, 0.0, 0.0, 0...
 * /ukf_se/alpha: 0.001
 * /ukf_se/base_link_frame: camera_link
 * /ukf_se/beta: 2
 * /ukf_se/control_config: [True, False, Fal...
 * /ukf_se/control_timeout: 0.2
 * /ukf_se/debug: False
 * /ukf_se/debug_out_file: /path/to/debug/fi...
 * /ukf_se/deceleration_gains: [1.0, 0.0, 0.0, 0...
 * /ukf_se/deceleration_limits: [1.3, 0.0, 0.0, 0...
 * /ukf_se/frequency: 300
 * /ukf_se/imu0: /imu/data
 * /ukf_se/imu0_config: [False, False, Fa...
 * /ukf_se/imu0_differential: True
 * /ukf_se/imu0_linear_acceleration_rejection_threshold: 0.8
 * /ukf_se/imu0_nodelay: False
 * /ukf_se/imu0_pose_rejection_threshold: 0.8
 * /ukf_se/imu0_queue_size: 5
 * /ukf_se/imu0_relative: False
 * /ukf_se/imu0_remove_gravitational_acceleration: True
 * /ukf_se/imu0_twist_rejection_threshold: 0.8
 * /ukf_se/initial_estimate_covariance: ['1e-9', 0, 0, 0,...
 * /ukf_se/kappa: 0
 * /ukf_se/map_frame: map
 * /ukf_se/odom0: rtabmap/odom
 * /ukf_se/odom0_config: [True, True, True...
 * /ukf_se/odom0_differential: False
 * /ukf_se/odom0_nodelay: False
 * /ukf_se/odom0_pose_rejection_threshold: 10000000
 * /ukf_se/odom0_queue_size: 2
 * /ukf_se/odom0_relative: True
 * /ukf_se/odom0_twist_rejection_threshold: 10000000
 * /ukf_se/odom1: example/another_odom
 * /ukf_se/odom1_config: [False, False, Tr...
 * /ukf_se/odom1_differential: False
 * /ukf_se/odom1_nodelay: False
 * /ukf_se/odom1_pose_rejection_threshold: 2
 * /ukf_se/odom1_queue_size: 2
 * /ukf_se/odom1_relative: True
 * /ukf_se/odom1_twist_rejection_threshold: 0.2
 * /ukf_se/odom_frame: odom
 * /ukf_se/pose0: example/pose
 * /ukf_se/pose0_config: [True, True, Fals...
 * /ukf_se/pose0_differential: True
 * /ukf_se/pose0_nodelay: False
 * /ukf_se/pose0_queue_size: 5
 * /ukf_se/pose0_rejection_threshold: 2
 * /ukf_se/pose0_relative: False
 * /ukf_se/print_diagnostics: True
 * /ukf_se/process_noise_covariance: [0.05, 0, 0, 0, 0...
 * /ukf_se/sensor_timeout: 0.1
 * /ukf_se/stamped_control: False
 * /ukf_se/transform_time_offset: 0.0
 * /ukf_se/transform_timeout: 0.0
 * /ukf_se/twist0: example/twist
 * /ukf_se/twist0_config: [False, False, Fa...
 * /ukf_se/twist0_nodelay: False
 * /ukf_se/twist0_queue_size: 3
 * /ukf_se/twist0_rejection_threshold: 2
 * /ukf_se/two_d_mode: False
 * /ukf_se/use_control: False
 * /ukf_se/world_frame: odom

NODES
  /
    ImuFilter (imu_filter_madgwick/imu_filter_node)
    points_xyzrgb (nodelet/nodelet)
    rviz (rviz/rviz)
    ukf_se (robot_localization/ukf_localization_node)
  /camera/
    realsense2_camera (nodelet/nodelet)
    realsense2_camera_manager (nodelet/nodelet)
  /rtabmap/
    rgbd_odometry (rtabmap_ros/rgbd_odometry)
    rtabmap (rtabmap_ros/rtabmap)

ROS_MASTER_URI=http://localhost:11311

[MartyG-RealSense]

Hi @Honza0297 A noticable difference between your setup and that of the guide is that you have everything in your Frames list ticked, whilst the guide says to only tick camera_link and map.

https://github.com/IntelRealSense/realsense-ros/wiki/SLAM-with-D435i#personalize-rviz

Once you have ticked those two boxes, you should go to Image, tick the box beside it and set the topic shown below after it has been ticked:

/camera/color/image_raw

[Honza0297]

@MartyG-RealSense Thanks for the answer! However, I have no box named "map" in my Frames list which i forgot to explicitly mention, sorry for that.

[MartyG-RealSense]

Np problem. Can you confirm that you are using the roslaunch instruction stated by the guide please:

roslaunch realsense2_camera opensource_tracking.launch

[Honza0297]

I can confirm that.

[MartyG-RealSense]

Let's go through the setup process step by step. Next, you can confirm that realsense2_camera (the RealSense ROS wrapper) is working correctly and you have installed imu_filter_madgwick, rtabmap_ros and robot_localization ?

https://github.com/IntelRealSense/realsense-ros/wiki/SLAM-with-D435i#installation

[Honza0297]

I can confirm all components are installed. And as far as I can tell,the RealSense ROS wrapper is working. I can see pointcloud output from the camera etc., so it should work.

[MartyG-RealSense]

@Honza0297 I have been over everything and I cannot see a reason why it is not working. There are a small number of cases where the D435i SLAM guide does not work and there is no reason why that can be found. Like @acarrou found when it worked on one computer but not the other.

[Honza0297]

OK, I think it can be related to my use case - Raspberry Pi 4, Ubuntu Mate 20.04 and ROS noetic. I will try to reinstall everything once again and once it is done, I'll post the state of things. Anyway, thank you for your help! :)

[Honza0297]

Unfortunately, reinstallation did not help. I can still see the pointcloud, but I have no map box under my Frames and I still cannot map the area (the pointcloud I see is only the one that is currently "seen" by the camera).

[MartyG-RealSense]

@Honza0297 I note that you have a red stop sign icon beside Pointcloud2 in your RViz panel. I did further research into past RealSense ROS cases where people could not get Pointcloud2 successfully,

In the bottom corner of the side-panel, there should be an Add button.

A RealSense user suggested the following procedure to access Pointcloud2:

1. Click on the Add button.
2. Click on the By topic tab.
3. Choose pointcloud2 under 'depth registered' >> 'points' abd click OK

[Honza0297]

Hello and sorry for the late reply. I am unable to test it in now because I don't have Raspberry with me, but i will test it and tell the result during this week. Thanks again!

[MartyG-RealSense]

Thanks very much - I look forward to hearing about your results!

[MartyG-RealSense]

Adding a note to keep this case open for a further time period.

[Honza0297]

Apologies for long response time, many things needed to be done (but I should answer more promptly now). I tried your advice, but no change. I still cannot figure out the reason why I am not seeing the box named map under my Frames and cannot map the area.

[MartyG-RealSense]

Hi @Honza0297 Are you using ROS Melodic like @acarrou was?

Edit: No, I read through the case again and see that you are using Noetic. I wonder if installing the kinetic versions of the required components suggested by the D435i SLAM guide with the much newer Noetic may be causing issues.

[MartyG-RealSense]

i @acarrou Do you require further assistance with this case, please? Thanks!

[acarrou]

nope thank you for your help! @MartyG-RealSense

[MartyG-RealSense]

Thanks very much @acarrou for the update!

[amineKourta]

I have the same problem as @Honza0297 no map box to ckeck in TF. I'm on Ubuntu 20.04 with noetic. I followed the guide to install requirements and made sure to change 'kinetic' into 'noetic' everytime. The installation worked properly. But I can't even see the current pointcloud. When I listen to the topic /camera/imu there's no messages

[MartyG-RealSense]

Hi @amineKourta Are you using the D435i SLAM guide please?

https://github.com/IntelRealSense/realsense-ros/wiki/SLAM-with-D435i

If so, you should ensure that the IMU topics are enabled so that they can be published, because they have been disabled by default since ROS wrapper version 2.2.15.

To enable the IMU topics if you have not done so already, please add the commands below to the end of your roslaunch instruction:

enable_accel:=true enable_gyro:=true

[amineKourta]

Hi @MartyG-RealSense thanks for your response. Yes I'm using that guide and I actually get the topic and no messages when I add enable_accel:=true enable_gyro:=true. If I don't add it I don't get an imu topic at all.

I solved the problem and was gonna write a reply with the response. I changed the rs_camera.launch and changed the following args to true : enable_infra, enable_infra1, enable_infra2, enable_gyro, enable_accel and now everything works right out of the box like the guide.

[MartyG-RealSense]

Thanks very much for the update - great news that you got it working!

[ozumipl]

nope thank you for your help! @MartyG-RealSense

Hi @acarrou , now I have been fighting with same issue that there is no map in the list in rviz. So, did you able to solve it? If soi is there any update? Thanks.

[MartyG-RealSense]

Hi @ozumipl In the particular case of @acarrou - they mentioned at https://github.com/IntelRealSense/realsense-ros/issues/1518#issuecomment-735352301 that enabling the accel, gyro and infra topics worked on their desktop computer

[ozumipl]

Hi @ozumipl In the particular case of @acarrou - they mentioned at #1518 (comment) that enabling the accel, gyro and infra topics worked on their desktop computer

Ohw, pretty good. sorry for that. But; my computer is now Jetson Nano 4GB, I am able to receive point cloud data and camera image topics properly but not able to select map from the list unfortunately. Btw, this will be a companion computer of the drone, currently I do not have T265 camera, I need to try it with only D435i. It gives an error such as "Fixed frame (map) does not ex ist.

[MartyG-RealSense]

@ozumipl If you are using the ROS1 wrapper then you can generate a point cloud by using the roslaunch command roslaunch realsense2_camera rs_camera.launch filters:=pointcloud and setting Fixed Frame to camera_link instead of map.

https://github.com/IntelRealSense/realsense-ros#point-cloud