rtabmap cannot function properly with rosbag

[AddyPete]

ROS2 Version: Humble RTABMap Version: RTABMap ROS2 Latest

Video Link

Mapping with RTABMap works properly and smoothly with Zed Wrapper when I do it live.

However when I try to do mapping with the rosbag recorded with zed wrapper, it can't function properly.

During the recording process I did this steps:

1. ros2 launch zed_wrapper zed_camera.launch.py camera_model:=zed2i

   • to launch the zed wrapper

2. ros2 bag record -a

   • to record all topics.

After recording, I tried mapping with the saved rosbag with: 1.

ros2 launch rtabmap_launch rtabmap.launch.py \
    rtabmap_args:="--delete_db_on_start" \
    rgb_topic:=/zed/zed_node/rgb/image_rect_color \
    depth_topic:=/zed/zed_node/depth/depth_registered \
    camera_info_topic:=/zed/zed_node/rgb/camera_info \
    frame_id:=zed_camera_center \
    approx_sync:=false \
    wait_imu_to_init:=true \
    imu_topic:=/zed/zed_node/imu/data \
    qos:=1 \
    use_sim_time:=true \
    rviz:=true

Then play the rosbag

2. ros2 bag play zed_office_test_1/zed_office_test_1_0.db3

However, mapping won't properly function with errors given such as TF_OLD_DATA and imu_link error messages. I also tried not adding use_sim_time:=false, but still won't work properly. You can check the YouTube Video Link for reference. Thank you!

[AddyPete]

I also tried adding --clock flag in rosbag at the same time adding use_sim_time:=true in rtabmap launch

but it produces another kind of error

[rgbd_odometry-1] [ERROR] [1700787162.614928777] [rtabmap.rgbd_odometry]: Could not transform IMU msg from frame "zed_imu_link" to frame "zed_camera_center", TF not available at time 1700553808.332129
[rgbd_odometry-1] [ERROR] [1700787163.368926398] [rtabmap.rgbd_odometry]: Could not transform IMU msg from frame "zed_imu_link" to frame "zed_camera_center", TF not available at time 1700553809.046681
[rgbd_odometry-1] [INFO] [1700787163.608385196] [rtabmap.rgbd_odometry]: Odom: quality=0, std dev=0.000000m|0.000000rad, update time=0.050037s
[rtabmap-2] [INFO] [1700787163.609433973] [rtabmap.rtabmap]: rtabmap (1): Rate=1.00s, Limit=0.000s, Conversion=0.0007s, RTAB-Map=0.0001s, Maps update=0.0000s pub=0.0000s (local map=1, WM=1)
[rgbd_odometry-1] [ERROR] [1700787164.196360636] [rtabmap.rgbd_odometry]: Could not transform IMU msg from frame "zed_imu_link" to frame "zed_camera_center", TF not available at time 1700553809.917001
[rgbd_odometry-1] [ERROR] [1700787164.621871336] [rtabmap.rgbd_odometry]: Could not transform IMU msg from frame "zed_imu_link" to frame "zed_camera_center", TF not available at time 1700553810.317546
[rgbd_odometry-1] [ERROR] [1700787164.814506473] [rtabmap.rgbd_odometry]: Could not transform IMU msg from frame "zed_imu_link" to frame "zed_camera_center", TF not available at time 1700553810.532654
[rgbd_odometry-1] [ERROR] [1700787165.645171164] [rtabmap.rgbd_odometry]: Could not transform IMU msg from frame "zed_imu_link" to frame "zed_camera_center", TF not available at time 1700553811.341161

[AddyPete]

I also tried frame_id:=zed_camera_link instead of zed_camera_center

it maps, but still laggy and odom will lost eventually. It has also errors:

[rgbd_odometry-1] [ERROR] [1700787376.131512555] [rtabmap.rgbd_odometry]: Could not transform IMU msg from frame "zed_imu_link" to frame "zed_camera_link", TF not available at time 1700553820.247576
[rgbd_odometry-1] [ERROR] [1700787376.708945620] [rtabmap.rgbd_odometry]: Could not transform IMU msg from frame "zed_imu_link" to frame "zed_camera_link", TF not available at time 1700553820.800856
[rgbd_odometry-1] [ERROR] [1700787377.906563152] [rtabmap.rgbd_odometry]: Could not transform IMU msg from frame "zed_imu_link" to frame "zed_camera_link", TF not available at time 1700553821.986814
[rgbd_odometry-1] [ERROR] [1700787378.109684955] [rtabmap.rgbd_odometry]: Could not transform IMU msg from frame "zed_imu_link" to frame "zed_camera_link", TF not available at time 1700553822.221701
[rgbd_odometry-1] [ERROR] [1700787378.688240381] [rtabmap.rgbd_odometry]: Could not transform IMU msg from frame "zed_imu_link" to frame "zed_camera_link", TF not available at time 1700553822.797794
[rgbd_odometry-1] [WARN] [1700787378.688394028] [rtabmap.rgbd_odometry]: rgbd_odometry: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called.
[rgbd_odometry-1] rgbd_odometry subscribed to (exact sync):
[rgbd_odometry-1]    /zed/zed_node/rgb/image_rect_color,
[rgbd_odometry-1]    /zed/zed_node/depth/depth_registered,
[rgbd_odometry-1]    /zed/zed_node/rgb/camera_info
[rgbd_odometry-1] [ERROR] [1700787379.063363964] [rtabmap.rgbd_odometry]: Could not transform IMU msg from frame "zed_imu_link" to frame "zed_camera_link", TF not available at time 1700553823.203284
[rgbd_odometry-1] [ERROR] [1700787379.438308625] [rtabmap.rgbd_odometry]: Could not transform IMU msg from frame "zed_imu_link" to frame "zed_camera_link", TF not available at time 1700553823.579104
[rgbd_odometry-1] [INFO] [1700787379.519100601] [rtabmap.rgbd_odometry]: Odom: quality=0, std dev=0.000000m|0.000000rad, update time=0.075001s
[rtabmap-2] [WARN] [1700787379.632034209] [rtabmap.rtabmap]: rtabmap: Did not receive data since 5 seconds! Make sure the input topics are published ("$ ros2 topic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called.
[rtabmap-2] rtabmap subscribed to (exact sync):
[rtabmap-2]    /rtabmap/odom \
[rtabmap-2]    /zed/zed_node/rgb/image_rect_color \
[rtabmap-2]    /zed/zed_node/depth/depth_registered \
[rtabmap-2]    /zed/zed_node/rgb/camera_info \
[rtabmap-2]    /rtabmap/odom_info
[rtabmap_viz-3] [WARN] [1700787379.776429792] [rtabmap.rtabmap_viz]: rtabmap_viz: Did not receive data since 5 seconds! Make sure the input topics are published ("$ ros2 topic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called.
[rtabmap_viz-3] rtabmap_viz subscribed to (exact sync):

[matlabbe]

If you do ros2 bag info, are the number of images recorded make sense for the length of the bag? (If we expect 30 Hz frames, for 10 seconds bag you should see 300 msgs for each camera topic).

I would not recommend doing ros2 bag record -a when there are publishers using image_transport because you will be recording the same images in all different compression plugins. This could make the computer not able to write fast enough, thus dropping frames.

[AddyPete]

I recorded for about 1 minute +, so I think the number if images does not match with the length of the recording.

Question:

1. should I use --use-sim-time flag in when doing ros2 bag record?
2. In rgbd mapping. should I only record the topic

    /zed/zed_node/rgb/image_rect_color
    /zed/zed_node/depth/depth_registered
    /zed/zed_node/rgb/camera_info
    /zed/zed_node/imu/data

? Only this four topics needed right when recording in a ros2 bag?

[matlabbe]

Note that to avoid TF conflict with odom and map frames, disable them when launching zed wrapper. Just tried with:

ros2 launch zed_wrapper zed_camera.launch.py camera_model:=zed2 publish_tf:=false publish_map_tf:=false

ros2 bag record /tf /tf_static \
  /zed/zed_node/rgb/image_rect_color \
  /zed/zed_node/depth/depth_registered \
  /zed/zed_node/rgb/camera_info \
  /zed/zed_node/imu/data

The mapping would work whether you set use_sim_time to true or false, however without use_sim_time, you may see warnings that rtabmap is not receiving data, but it does.

ros2 launch rtabmap_launch rtabmap.launch.py  \
   rtabmap_args:="--delete_db_on_start"  \
   rgb_topic:=/zed/zed_node/rgb/image_rect_color  \
   depth_topic:=/zed/zed_node/depth/depth_registered  \
   camera_info_topic:=/zed/zed_node/rgb/camera_info  \
   frame_id:=zed_camera_link  
   approx_sync:=false  \
   wait_imu_to_init:=true  \
   imu_topic:=/zed/zed_node/imu/data  \
   qos:=1  \
   use_sim_time:=true

I got also this issue (my computer is on Foxy), but I solved it by remapping /tf_static to /tf:

ros2 bag play rosbag2_2023_11_23-18_42_35_0.db3 --remap /tf_static:=/tf

[AddyPete]

thank you very much. it works now!