No image with realsense_d435i_color.launch.py

[ryanmaxwell96]

I'm having issues getting an image to appear when running the following command:

ros2 launch rtabmap_ros realsense_d435i_color.launch.py

My setup is:

Jetson AGX Xavier Jetpack 5.0.2 Ubuntu 20.04 Foxy ROS2 distro Realsense camera d435i

The terminal output that comes is this:

`[INFO] [launch]: All log files can be found below /home/nvidia/.ros/log/2022-12-09-13-51-30-676381-nvidia-desktop-12755 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [rgbd_odometry-1]: process started with pid [12757] [INFO] [rtabmap-2]: process started with pid [12759] [INFO] [rtabmapviz-3]: process started with pid [12761] [INFO] [point_cloud_xyz-4]: process started with pid [12763] [INFO] [pointcloud_to_depthimage-5]: process started with pid [12765] [INFO] [imu_filter_madgwick_node-6]: process started with pid [12767] [INFO] [static_transform_publisher-7]: process started with pid [12769] [static_transform_publisher-7] [INFO] [1670626292.087608462] [static_transform_publisher_2Thqt8CnRIyfq9Ie]: Spinning until killed publishing transform from 'camera_gyro_optical_frame' to 'camera_imu_optical_frame' [imu_filter_madgwick_node-6] [INFO] [1670626292.105999728] [imu_filter_madgwick]: Starting ImuFilter [imu_filter_madgwick_node-6] [INFO] [1670626292.121180900] [imu_filter_madgwick]: Using dt computed from message headers [imu_filter_madgwick_node-6] [INFO] [1670626292.122127639] [imu_filter_madgwick]: The gravity vector is kept in the IMU message. [imu_filter_madgwick_node-6] [INFO] [1670626292.127043169] [imu_filter_madgwick]: Imu filter gain set to 0.100000 [imu_filter_madgwick_node-6] [INFO] [1670626292.127545916] [imu_filter_madgwick]: Gyro drift bias set to 0.000000 [imu_filter_madgwick_node-6] [INFO] [1670626292.127791625] [imu_filter_madgwick]: Magnetometer bias values: 0.000000 0.000000 0.000000

[point_cloud_xyz-4] [INFO] [1670626294.895261189] [point_cloud_xyz]: Approximate time sync = false

[pointcloud_to_depthimage-5] [INFO] [1670626294.898806243] [pointcloud_to_depthimage]: fill_holes_size=1 pixels (0=disabled)

[pointcloud_to_depthimage-5] [WARN] [1670626294.903263090] [rcl.logging_rosout]: Publisher already registered for provided node name. If this is due to multiple nodes with the same name then all logs for that logger name will go out over the existing publisher. As soon as any node with that name is destructed it will unregister the publisher, preventing any further logs for that name from being published on the rosout topic. [rgbd_odometry-1] [INFO] [1670626295.114063503] [rgbd_odometry]: Odometry: frame_id = camera_link [rgbd_odometry-1] [INFO] [1670626295.125191521] [rgbd_odometry]: Odometry: odom_frame_id = odom [rgbd_odometry-1] [INFO] [1670626295.125549012] [rgbd_odometry]: Odometry: publish_tf = true [rgbd_odometry-1] [INFO] [1670626295.125669306] [rgbd_odometry]: Odometry: wait_for_transform = 0.100000 [rgbd_odometry-1] [INFO] [1670626295.126133747] [rgbd_odometry]: Odometry: initial_pose = xyz=0.000000,0.000000,0.000000 rpy=0.000000,-0.000000,0.000000 [rgbd_odometry-1] [INFO] [1670626295.126238712] [rgbd_odometry]: Odometry: ground_truth_frame_id = [rgbd_odometry-1] [INFO] [1670626295.127046851] [rgbd_odometry]: Odometry: ground_truth_base_frame_id = camera_link [rgbd_odometry-1] [INFO] [1670626295.127182539] [rgbd_odometry]: Odometry: config_path = [rgbd_odometry-1] [INFO] [1670626295.127254639] [rgbd_odometry]: Odometry: publish_null_when_lost = true [rgbd_odometry-1] [INFO] [1670626295.127324722] [rgbd_odometry]: Odometry: guess_frame_id = [rgbd_odometry-1] [INFO] [1670626295.127596929] [rgbd_odometry]: Odometry: guess_min_translation = 0.000000 [rgbd_odometry-1] [INFO] [1670626295.127701382] [rgbd_odometry]: Odometry: guess_min_rotation = 0.000000 [rgbd_odometry-1] [INFO] [1670626295.127774378] [rgbd_odometry]: Odometry: guess_min_time = 0.000000 [rgbd_odometry-1] [INFO] [1670626295.127843566] [rgbd_odometry]: Odometry: expected_update_rate = 0.000000 Hz [rgbd_odometry-1] [INFO] [1670626295.127916530] [rgbd_odometry]: Odometry: max_update_rate = 0.000000 Hz [rgbd_odometry-1] [INFO] [1670626295.130965364] [rgbd_odometry]: Odometry: wait_imu_to_init = true [rgbd_odometry-1] [INFO] [1670626295.131415020] [rgbdodometry]: Odometry: stereoParams=0 visParams=1 icpParams=0 [rtabmap-2] [INFO] [1670626295.261323610] [rtabmap]: rtabmap(maps): map_filter_radius = 0.000000 [rtabmap-2] [INFO] [1670626295.264712046] [rtabmap]: rtabmap(maps): map_filter_angle = 30.000000 [rtabmap-2] [INFO] [1670626295.265086466] [rtabmap]: rtabmap(maps): map_cleanup = true [rtabmap-2] [INFO] [1670626295.265442965] [rtabmap]: rtabmap(maps): map_always_update = false [rtabmap-2] [INFO] [1670626295.265581789] [rtabmap]: rtabmap(maps): map_empty_ray_tracing = true [rtabmap-2] [INFO] [1670626295.265679138] [rtabmap]: rtabmap(maps): cloud_output_voxelized = true [rtabmap-2] [INFO] [1670626295.265763558] [rtabmap]: rtabmap(maps): cloud_subtract_filtering = false [rtabmap-2] [INFO] [1670626295.265956721] [rtabmap]: rtabmap(maps): cloud_subtract_filtering_min_neighbors = 2 [rtabmap-2] [INFO] [1670626295.266587218] [rtabmap]: rtabmap(maps): octomap_tree_depth = 16 [rgbd_odometry-1] [INFO] [1670626295.435866994] [rgbd_odometry]: odometry: Subscribing to IMU topic /imu/data [rgbd_odometry-1] [INFO] [1670626295.436400143] [rgbd_odometry]: odometry: qos_imu = 0 [rgbd_odometry-1] [INFO] [1670626295.472253798] [rgbd_odometry]: RGBDOdometry: approx_sync = false [rgbd_odometry-1] [INFO] [1670626295.476467719] [rgbd_odometry]: RGBDOdometry: queue_size = 5 [rgbd_odometry-1] [INFO] [1670626295.476904382] [rgbd_odometry]: RGBDOdometry: qos = 0 [rgbd_odometry-1] [INFO] [1670626295.477115273] [rgbd_odometry]: RGBDOdometry: qos_camera_info = 0 [rgbd_odometry-1] [INFO] [1670626295.477378967] [rgbd_odometry]: RGBDOdometry: subscribe_rgbd = false [rgbd_odometry-1] [INFO] [1670626295.477508862] [rgbd_odometry]: RGBDOdometry: rgbd_cameras = 1 [rgbd_odometry-1] [INFO] [1670626295.477621700] [rgbd_odometry]: RGBDOdometry: keep_color = false [rgbd_odometry-1] [INFO] [1670626295.556833315] [rgbd_odometry]: [rgbd_odometry-1] rgbd_odometry subscribed to (exact sync): [rgbd_odometry-1] /camera/color/image_raw, [rgbd_odometry-1] /camera/realigned_depth_to_color/image_raw, [rgbd_odometry-1] /camera/color/camera_info [rtabmap-2] [INFO] [1670626295.569380032] [rtabmap]: rtabmap: frame_id = camera_link [rtabmap-2] [INFO] [1670626295.569931421] [rtabmap]: rtabmap: map_frame_id = map [rtabmap-2] [INFO] [1670626295.570126632] [rtabmap]: rtabmap: initial_pose = [rtabmap-2] [INFO] [1670626295.570237966] [rtabmap]: rtabmap: use_action_for_goal = false [rtabmap-2] [INFO] [1670626295.570578528] [rtabmap]: rtabmap: tf_delay = 0.050000 [rtabmap-2] [INFO] [1670626295.570738504] [rtabmap]: rtabmap: tf_tolerance = 0.100000 [rtabmap-2] [INFO] [1670626295.570858447] [rtabmap]: rtabmap: odom_sensor_sync = false [rtabmap-2] [INFO] [1670626295.570936755] [rtabmap]: rtabmap: gen_scan = false [rtabmap-2] [INFO] [1670626295.571004950] [rtabmap]: rtabmap: gen_depth = false [rtabmap-2] [INFO] [1670626296.139080545] [rtabmap]: RTAB-Map detection rate = 1.000000 Hz [rtabmap-2] [INFO] [1670626296.166053465] [rtabmap]: rtabmap: Deleted database "/home/nvidia/.ros/rtabmap.db" (--delete_db_on_start or -d are set). [rtabmap-2] [INFO] [1670626296.166815265] [rtabmap]: rtabmap: Using database from "/home/nvidia/.ros/rtabmap.db" (0 MB). [rtabmap-2] [INFO] [1670626296.488093820] [rtabmap]: rtabmap: Database version = "0.20.22". [rtabmap-2] [INFO] [1670626296.488910823] [rtabmap]: rtabmap: SLAM mode (Mem/IncrementalMemory=true) [rtabmap-2] [INFO] [1670626296.753523298] [rtabmap]: Setup callbacks [rtabmap-2] [INFO] [1670626296.755480234] [rtabmap]: rtabmap: subscribe_depth = true [rtabmap-2] [INFO] [1670626296.755873471] [rtabmap]: rtabmap: subscribe_rgb = true [rtabmap-2] [INFO] [1670626296.756033319] [rtabmap]: rtabmap: subscribe_stereo = false [rtabmap-2] [INFO] [1670626296.756141421] [rtabmap]: rtabmap: subscribe_rgbd = false (rgbd_cameras=1) [rtabmap-2] [INFO] [1670626296.756261907] [rtabmap]: rtabmap: subscribe_odom_info = false [rtabmap-2] [INFO] [1670626296.756467390] [rtabmap]: rtabmap: subscribe_user_data = false [rtabmap-2] [INFO] [1670626296.756570884] [rtabmap]: rtabmap: subscribe_scan = false [rtabmap-2] [INFO] [1670626296.756649928] [rtabmap]: rtabmap: subscribe_scan_cloud = false [rtabmap-2] [INFO] [1670626296.756721516] [rtabmap]: rtabmap: subscribe_scan_descriptor = false [rtabmap-2] [INFO] [1670626296.756793903] [rtabmap]: rtabmap: queue_size = 10 [rtabmap-2] [INFO] [1670626296.756868243] [rtabmap]: rtabmap: qos_image = 0 [rtabmap-2] [INFO] [1670626296.756935287] [rtabmap]: rtabmap: qos_camera_info = 0 [rtabmap-2] [INFO] [1670626296.756998810] [rtabmap]: rtabmap: qos_scan = 0 [rtabmap-2] [INFO] [1670626296.757063262] [rtabmap]: rtabmap: qos_odom = 0 [rtabmap-2] [INFO] [1670626296.757142466] [rtabmap]: rtabmap: qos_user_data = 0 [rtabmap-2] [INFO] [1670626296.757206661] [rtabmap]: rtabmap: approx_sync = false [rtabmap-2] [INFO] [1670626296.757442066] [rtabmap]: Setup depth callback [rtabmap-2] [INFO] [1670626296.820137873] [rtabmap]: [rtabmap-2] rtabmap subscribed to (exact sync): [rtabmap-2] /odom \ [rtabmap-2] /camera/color/image_raw \ [rtabmap-2] /camera/realigned_depth_to_color/image_raw \ [rtabmap-2] /camera/color/camera_info [rtabmapviz-3] [INFO] [1670626297.639728129] [rtabmapviz]: rtabmapviz: Using configuration from "/home/nvidia/.ros/rtabmapGUI.ini" [rgbd_odometry-1] [WARN] [1670626300.562214923] [rgbd_odometry]: rgbd_odometry: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called. [rgbd_odometry-1] rgbd_odometry subscribed to (exact sync): [rgbd_odometry-1] /camera/color/image_raw, [rgbd_odometry-1] /camera/realigned_depth_to_color/image_raw, [rgbd_odometry-1] /camera/color/camera_info [rtabmapviz-3] libpng warning: iCCP: known incorrect sRGB profile [rtabmapviz-3] libpng warning: iCCP: known incorrect sRGB profile [rtabmapviz-3] libpng warning: iCCP: known incorrect sRGB profile [rtabmap-2] [WARN] [1670626301.823472789] [rtabmap]: rtabmap: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). If topics are not published at the same rate, you could increase "queue_size" parameter (current=10). Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called. [rtabmap-2] rtabmap subscribed to (exact sync): [rtabmap-2] /odom \ [rtabmap-2] /camera/color/image_raw \ [rtabmap-2] /camera/realigned_depth_to_color/image_raw \ [rtabmap-2] /camera/color/camera_info [imu_filter_madgwick_node-6] [WARN] [1670626302.231311337] [imu_filter_madgwick]: Still waiting for data on topic /imu/data_raw... [rtabmapviz-3] [WARN] [1670626302.400081047] [rcl.logging_rosout]: Publisher already registered for provided node name. If this is due to multiple nodes with the same name then all logs for that logger name will go out over the existing publisher. As soon as any node with that name is destructed it will unregister the publisher, preventing any further logs for that name from being published on the rosout topic. [rtabmapviz-3] [INFO] [1670626302.466770924] [rtabmapviz]: Reading parameters from the ROS server... [rtabmapviz-3] [INFO] [1670626302.832014942] [rtabmapviz]: Parameters read = 359 [rtabmapviz-3] [INFO] [1670626302.832908908] [rtabmapviz]: Parameters successfully read. [rtabmapviz-3] [INFO] [1670626304.784003428] [rtabmapviz]: rtabmapviz: subscribe_depth = true [rtabmapviz-3] [INFO] [1670626304.784431546] [rtabmapviz]: rtabmapviz: subscribe_rgb = false [rtabmapviz-3] [INFO] [1670626304.784540672] [rtabmapviz]: rtabmapviz: subscribe_stereo = false [rtabmapviz-3] [INFO] [1670626304.784591682] [rtabmapviz]: rtabmapviz: subscribe_rgbd = false (rgbd_cameras=1) [rtabmapviz-3] [INFO] [1670626304.784639557] [rtabmapviz]: rtabmapviz: subscribe_odom_info = false [rtabmapviz-3] [INFO] [1670626304.784685287] [rtabmapviz]: rtabmapviz: subscribe_user_data = false [rtabmapviz-3] [INFO] [1670626304.784729897] [rtabmapviz]: rtabmapviz: subscribe_scan = false [rtabmapviz-3] [INFO] [1670626304.784768779] [rtabmapviz]: rtabmapviz: subscribe_scan_cloud = false [rtabmapviz-3] [INFO] [1670626304.784804877] [rtabmapviz]: rtabmapviz: subscribe_scan_descriptor = false [rtabmapviz-3] [INFO] [1670626304.784959157] [rtabmapviz]: rtabmapviz: queue_size = 10 [rtabmapviz-3] [INFO] [1670626304.785009912] [rtabmapviz]: rtabmapviz: qos_image = 0 [rtabmapviz-3] [INFO] [1670626304.785049082] [rtabmapviz]: rtabmapviz: qos_camera_info = 0 [rtabmapviz-3] [INFO] [1670626304.785091612] [rtabmapviz]: rtabmapviz: qos_scan = 0 [rtabmapviz-3] [INFO] [1670626304.785131070] [rtabmapviz]: rtabmapviz: qos_odom = 0 [rtabmapviz-3] [INFO] [1670626304.785168448] [rtabmapviz]: rtabmapviz: qos_user_data = 0 [rtabmapviz-3] [INFO] [1670626304.785212354] [rtabmapviz]: rtabmapviz: approx_sync = false [rtabmapviz-3] [INFO] [1670626304.785991402] [rtabmapviz]: Setup depth callback [rtabmapviz-3] [INFO] [1670626304.865481327] [rtabmapviz]: [rtabmapviz-3] rtabmapviz subscribed to (exact sync): [rtabmapviz-3] /odom \ [rtabmapviz-3] /camera/color/image_raw \ [rtabmapviz-3] /camera/realigned_depth_to_color/image_raw \ [rtabmapviz-3] /camera/color/camera_info [rtabmapviz-3] [INFO] [1670626304.867377488] [rtabmapviz]: rtabmapviz started. [rgbd_odometry-1] [WARN] [1670626305.562211070] [rgbd_odometry]: rgbd_odometry: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called. [rgbd_odometry-1] rgbd_odometry subscribed to (exact sync): [rgbd_odometry-1] /camera/color/image_raw, [rgbd_odometry-1] /camera/realigned_depth_to_color/image_raw, [rgbd_odometry-1] /camera/color/camera_info [rtabmap-2] [WARN] [1670626306.823134335] [rtabmap]: rtabmap: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). If topics are not published at the same rate, you could increase "queue_size" parameter (current=10). Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called. [rtabmap-2] rtabmap subscribed to (exact sync): [rtabmap-2] /odom \ [rtabmap-2] /camera/color/image_raw \ [rtabmap-2] /camera/realigned_depth_to_color/image_raw \ [rtabmap-2] /camera/color/camera_info [rtabmapviz-3] [WARN] [1670626309.865949682] [rtabmapviz]: rtabmapviz: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). If topics are not published at the same rate, you could increase "queue_size" parameter (current=10). Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called. [rtabmapviz-3] rtabmapviz subscribed to (exact sync): [rtabmapviz-3] /odom \ [rtabmapviz-3] /camera/color/image_raw \ [rtabmapviz-3] /camera/realigned_depth_to_color/image_raw \ [rtabmapviz-3] /camera/color/camera_info [rgbd_odometry-1] [WARN] [1670626310.561885374] [rgbd_odometry]: rgbd_odometry: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called. [rgbd_odometry-1] rgbd_odometry subscribed to (exact sync): [rgbd_odometry-1] /camera/color/image_raw, [rgbd_odometry-1] /camera/realigned_depth_to_color/image_raw, [rgbd_odometry-1] /camera/color/camera_info [rtabmap-2] [WARN] [1670626311.823144563] [rtabmap]: rtabmap: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). If topics are not published at the same rate, you could increase "queue_size" parameter (current=10). Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called. [rtabmap-2] rtabmap subscribed to (exact sync): [rtabmap-2] /odom \ [rtabmap-2] /camera/color/image_raw \ [rtabmap-2] /camera/realigned_depth_to_color/image_raw \ [rtabmap-2] /camera/color/camera_info [imu_filter_madgwick_node-6] [WARN] [1670626312.231443351] [imu_filter_madgwick]: Still waiting for data on topic /imu/data_raw... [rtabmapviz-3] [WARN] [1670626314.866105794] [rtabmapviz]: rtabmapviz: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). If topics are not published at the same rate, you could increase "queue_size" parameter (current=10). Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called. [rtabmapviz-3] rtabmapviz subscribed to (exact sync): [rtabmapviz-3] /odom \ [rtabmapviz-3] /camera/color/image_raw \ [rtabmapviz-3] /camera/realigned_depth_to_color/image_raw \ [rtabmapviz-3] /camera/color/camera_info [rgbd_odometry-1] [WARN] [1670626315.561941084] [rgbd_odometry]: rgbd_odometry: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called. [rgbd_odometry-1] rgbd_odometry subscribed to (exact sync): [rgbd_odometry-1] /camera/color/image_raw, [rgbd_odometry-1] /camera/realigned_depth_to_color/image_raw, [rgbd_odometry-1] /camera/color/camera_info ^C[WARNING] [launch]: user interrupted with ctrl-c (SIGINT) ^X[static_transform_publisher-7] [INFO] [1670626315.770534634] [rclcpp]: signal_handler(signal_value=2)

[rgbd_odometry-1] [WARN] [1670626315.774815131] [rgbd_odometry]: rgbd_odometry: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called.

`

When I run ros2 topic info /camera/color/image_raw, I get: Publisher count: 0

I'm not sure why it is not publishing. (realsense_d400.launch.py and realsense_d435i_infra.launch.py behave the same)

[ryanmaxwell96]

I'm trying to do SLAM with the d435i realsense. Maybe there is a better package to do this other than rtabmap_ros? Like Isaac-Ros?

[MartyG-RealSense]

Hi @ryanmaxwell96 There is a known issue with the RealSense IMU not working with JetPack 5, as this JetPack version is not yet officially supported by the librealsense SDK at the time of writing this (depth and color does work with it). Any RealSense-compatible SLAM solution that makes use of the IMU is therefore likely to experience the same issue when used with JetPack 5 if it requires IMU topics to be published by the RealSense ROS wrapper.

I consulted my Intel RealSense colleagues about this yesterday and they informed me that JetPack 5 support is being worked on and will be in a future SDK release, though I do not have a specific time estimate to provide unfortunately.

[ryanmaxwell96]

Ok thanks for the info. Should this be an issue preventing anything from showing up in rtabmap viewer? Because the image_raw is not publishing which it wouldn't make sense to be dependent on imu.

At any rate, I thought imu was working ok because I've been able to hit the motion module toggle in realsense-viewer with no errors. Plus, in 2D, it clearly shows the info for how the camera is oriented

[MartyG-RealSense]

An rtabmap_ros user of ROS2 Foxy at https://github.com/introlab/rtabmap_ros/issues/527 who was using realsense_d400.launch.py had the same set of errors. There was not a solution reached in that case though.

My research did not find a solution elsewhere unfortunately. A more typical approach to using rtabmap_ros with RealSense cameras is for the RealSense ROS wrapper to be roslaunched first and then the rtabmap_ros launch instruction inputted secondly after the ROS wrapper launch has completed, like in the tutorial at the link below.

http://wiki.ros.org/rtabmap_ros/Tutorials/HandHeldMapping

[MartyG-RealSense]

Hi @ryanmaxwell96 Have you made any progress with your problem, please? Thanks!

[ryanmaxwell96]

I'm waiting on getting a Samsung nvme 256gb to expand memory. I think I'm unsure at the moment what ROS, what Jetpack, and what Ubuntu version to go with as I want SLAM with the d435i and Isaac ROS looks good, thus needing to expand memory from 32GB to 256gb.

I don't suppose you'd have a suggestion as for which packages I should download, to do SLAM via ROS with Jetson Xavier AGX and D435i?

[MartyG-RealSense]

A RealSense user with a D435i, ROS2 Foxy and an Xavier NX at https://github.com/IntelRealSense/realsense-ros/issues/2387 used slam_toolbox

https://github.com/SteveMacenski/slam_toolbox

[ryanmaxwell96]

Ok so on the ros rtabmap tutorial website you shared, what commands should I use?

I tried to run the commands:

ros2 realsense2_camera rs_launch.py

And then

ros2 launch rtabmap_ros rtabmap.launch.py \ ... everything else as written in tutorial for D435i

But I still get a blank Rtabmap screen as shown above. Should the D435i image show up in 3D Map in rtabmap?

[MartyG-RealSense]

If you launched the camera with ros2 launch rtabmap_ros rtabmap.launch.py and added "everything else as written in tutorial for D435i", do you mean the D435i instructions from the HandHeldMapping rtabmap_ros tutorial please?

If so then using unite_imu_method:=linear_interpolation does not work with the ros2_development wrapper. Instead, unite_imu_method's modes are set with a number. The linear_interpolation mode is '2'.

unite_imu_method:=2

And align_depth:=true in the ros2_development wrapper is now align_depth.enable:=true

[ryanmaxwell96]

Thanks again for being responsive.

I ran these in order in 3 separate terminals

ros2 launch realsense2_camera rs_launch.py \ align_depth.enable:=true \ unite_imu_method:=2 \ enable_gyro:=true \ enable_accel:=true

ros2 run imu_filter_madgwick imu_filter_madgwick_node \
--ros-args --remap _use_mag:=false \
--ros-args --remap _publish_tf:=false \
--ros-args --remap _world_frame:=enu \
--ros-args --remap /imu/data_raw:=/camera/imu \
--ros-args --remap /imu/data:=/rtabmap/imu

ros2 launch rtabmap_ros rtabmap.launch.py

imu_filter_madgwick_node gives warnings that imu topic is not being published as expected (hopefully after added support for imu for jetpack 5 hopefully in January(?) this will go away). rs_launch.py gives a warning that the D435i is running as if it was on a USB 2 but depth and rgb work fine in realsense viewer. And rtabmap.launch.py runs the same as before with nothing showing up including the image.

When I run RVIZ2, at the very least i can get the image.

[MartyG-RealSense]

Thanks very much for the update. Does the USB 2 behaviour become USB 3 if initial_reset:=true is added to the rs_launch.py launch instruction?

[ryanmaxwell96]

No unfortunately it stays as usb2.1. It reports 2.1 in realsense-viewer too. I thought it would be ok given depth and rgb work in the realsense-viewer. Im using the stock USB cable that came with Jetson AGX Xavier H01 development kit so not sure if this is the reason or if it is software related

[MartyG-RealSense]

Do you still have the original USB3 USB-C cable supplied with the D435i?

[ryanmaxwell96]

Do you still have the original USB3 USB-C cable supplied with the D435i?

No unfortunately it did not come with the camera when I bought it

[MartyG-RealSense]

Are you using a USB hub and have the camera plugged into it? If the cable is USB 3 but the hub is a USB 2 hub then the camera would be treated as a USB 2 device.

[ryanmaxwell96]

Are you using a USB hub and have the camera plugged into it? If the cable is USB 3 but the hub is a USB 2 hub then the camera would be treated as a USB 2 device.

My camera is plugged directly into the Jetson USB 3.0 port

[MartyG-RealSense]

Assuming that the stock cable supplied with the Jetson AGX Xavier H01 developer kit is USB 3 since the ports are USB 3, if the cable is not damaged (e.g through excessive bending) then that would shift the suspicion towards a compatibility issue with JetPack 5.

[ryanmaxwell96]

Assuming that the stock cable supplied with the Jetson AGX Xavier H01 developer kit is USB 3 since the ports are USB 3, if the cable is not damaged (e.g through excessive bending) then that would shift the suspicion towards a compatibility issue with JetPack 5.

I tried with a standard Apple USB to USB C cable (unsure whether 3.0 or 2.0), but it gave same results in the viewer.

At any rate, if it gives depth and rgb info just fine in the viewer, I find it hard to believe this is the reason why rtabmap will not show anything.

Perhaps there is a transform issue? In RVIZ, it does not give any other options other than map for the base link and global is not set

[MartyG-RealSense]

My knowledge of rtabmap_ros is admittedly limited. My research of your question though found a number of references that advised that in order to visualize a 3D point cloud in RViz, the MapCloud plugin for RViz should be added if you have not done so already, as advised at the top of Section2. Mapping Mode of the rtabmap_ros guide.

http://wiki.ros.org/rtabmap_ros#Map_Cloud_Display

I also recall that in order to render the data in RViz, rtabmapviz should be set to false and rviz to true, like in the opernsource_tracking launch file.

https://github.com/IntelRealSense/realsense-ros/blob/ros1-legacy/realsense2_camera/launch/opensource_tracking.launch#L23-L24

[ryanmaxwell96]

Ok thanks I got MapCloud up and running.

I still have the transform issues. Map is the only frame that is listed but it gives errors as not being found. I think I need something about needing a static transform right? I will need to look further into this.

[MartyG-RealSense]

I am not familiar with this subject in ROS2. I do apologize. Please do update me if you make progress in your investigation.

[ryanmaxwell96]

I was playing around with the tutorial

https://github.com/introlab/rtabmap/wiki/Kinect-mapping

And after changing frame to 640x480 at 30Hz (rtabmap -> window -> preferences -> source after running rs-enumerate-devices for noobs like me), I finally got my first map!

The blue dots represent how the camera was moving, but doesn't this mean that the IMU is working?

[ryanmaxwell96]

I think I will close out this issue as it has gone in many different rabbit trails, sorry about that! At least I have a map now. But if I find out any more about ROS2 transform issue, I will let you know. Thanks for your prompt communication!

[MartyG-RealSense]

It's no trouble at all. It's great to hear that you made significant progress! Thanks very much for the update, and please do share any new ROS2 transform insights that you discover. :)

[jsb9945]

Hello, I want to create a 3d map using d435i. My current environment is ubuntu 20.04 and ros2 foxy.

I ran this command

ros2 launch realsense2_camera rs_launch.py align_depth.enable:=true unite_imu_method:=2 enable_gyro:=true enable_accel:=true

i got error

[INFO] [launch]: All log files can be found below /home/csrl/.ros/log/2023-01-28-16-34-13-313569-csrl-System-Product-Name-33149 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [realsense2_camera_node-1]: process started with pid [33151] [realsense2_camera_node-1] [INFO] [1674891253.617941112] [RealSenseCameraNode]: RealSense ROS v3.2.2 [realsense2_camera_node-1] [INFO] [1674891253.618003537] [RealSenseCameraNode]: Built with LibRealSense v2.51.1 [realsense2_camera_node-1] [INFO] [1674891253.618015590] [RealSenseCameraNode]: Running with LibRealSense v2.51.1 [realsense2_camera_node-1] [INFO] [1674891253.661201263] [RealSenseCameraNode]: Device with serial number 943222072377 was found. [realsense2_camera_node-1] [realsense2_camera_node-1] [INFO] [1674891253.661252468] [RealSenseCameraNode]: Device with physical ID /sys/devices/pci0000:00/0000:00:01.3/0000:03:00.0/usb2/2-3/2-3:1.0/video4linux/video0 was found. [realsense2_camera_node-1] [INFO] [1674891253.661269339] [RealSenseCameraNode]: Device with name Intel RealSense D435I was found. [realsense2_camera_node-1] [INFO] [1674891253.661834474] [RealSenseCameraNode]: Device with port number 2-3 was found. [realsense2_camera_node-1] [INFO] [1674891253.661858348] [RealSenseCameraNode]: Device USB type: 3.2

[realsense2_camera_node-1] [ERROR] [1674891253.666404722] [RealSenseCameraNode]: Failed to set parameter: unite_imu_method. expected [string] got [integer] [realsense2_camera_node-1] expected [string] got [integer] [realsense2_camera_node-1] terminate called after throwing an instance of 'rclcpp::ParameterTypeException' [realsense2_camera_node-1] what(): expected [string] got [integer] [ERROR] [realsense2_camera_node-1]: process has died [pid 33151, exit code -6, cmd '/home/csrl/ros3_ws/install/realsense2_camera/lib/realsense2_camera/realsense2_camera_node --ros-args --log-level info --ros-args -r node:=camera -r ns:=/camera --params-file /tmp/launch_params_baofn84g'].

How can i solve it?

[MartyG-RealSense]

Hi @jsb9945 It appears that you are using some instructions that only work in the ros2-development ROS2 wrapper, whilst wrapper 3.2.2 belongs to a previous generation of ROS2 wrapper that has different commands and is now obsolete. The launch commands for these functions in 3.2.2 should be:

ros2 launch realsense2_camera rs_launch.py align_depth:=true enable_gyro:=true
enable_accel:=true unite_imu_method:=linear_interpolation

[jsb9945]

Hello. Thank you for your quick reply. Which version of the wrapper should I use?

[MartyG-RealSense]

The version of the current generation of the ROS2 wrapper that should be used with librealsense 2.51.1 is 4.51.1.

https://github.com/IntelRealSense/realsense-ros/releases/tag/4.51.1

However, if the previous generation wrapper 3.2.2 works for you and provides the results that you need then you may not need to change wrappers. You will have to move to the current wrapper generation if you use a librealsense version newer than 2.51.1 in future though.

[jsb9945]

Thank you very much. I solved problem!!

[MartyG-RealSense]

That's great to hear, @jsb9945 - thanks very much for the update!

[MrDarknessWolf]

question where did either realsense_d400.launch.py and the rtabmap.launch.py for rtabmap_ros went ? every time when executing the command it says its missing from the folder

[MartyG-RealSense]

Hi @MrDarknessWolf This launch file is still present in the 'rtabmap_examples' directory of the ros2 branch of rtabmap_ros but not in the same folder on the main master branch.

https://github.com/introlab/rtabmap_ros/tree/ros2/rtabmap_examples/launch