cannot start sync slam on a remote pc

[timbo-r]

Please provide the following information:

• OS: Ubuntu 20.04.05 LTS
• ROS Distro: Galactic
• Built from source or installed: installed according to the official documentation

• Package version: ros-galactic-turtlebot4-navigation (0.1.2-1focal.20220922.210710)

  Expected behaviour A map should be created and updated in rviz2

  Actual behaviour According to the turtlebot4 user manual I try to start slam_sync.launch.py on a remote pc, but I get the following error repeatedly:

[sync_slam_toolbox_node-1] [INFO] [1665068583.432561558] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665068582.845 for reason 'discarding message because the queue is full'

When executing view_robot.launch.py, in another terminal, the following errors are shown, repeatedly:

[rviz2-1] [INFO] [1665068580.660212631] [rviz2]: Message Filter dropping message: frame 'rplidar_link' at time 1665068576,397 for reason 'discarding message because the queue is full'
[rviz2-1] Warning: Invalid frame ID "wheel_drop_left" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "wheel_drop_right" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "wheel_drop_left" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "wheel_drop_right" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] [INFO] [1665068581.171618589] [rviz2]: Message Filter dropping message: frame 'rplidar_link' at time 1665068576,532 for reason 'discarding message because the queue is full'
[rviz2-1] Warning: Invalid frame ID "wheel_drop_left" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "wheel_drop_right" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "wheel_drop_left" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "wheel_drop_right" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] [INFO] [1665068581.612268664] [rviz2]: Message Filter dropping message: frame 'rplidar_link' at time 1665068577,060 for reason 'discarding message because the queue is full'

Starting sync SLAM on the Raspberry directly is creating the map as expected, with no errors.

To Reproduce Provide the steps to reproduce:

1. On the remote PC run

ros2 launch turtlebot4_navigation slam_sync.launch.py

2. On the remote PC, in another terminal, run

ros2 launch turtlebot4_viz view_robot.launch.py

3. see the error

Other notes

• When rviz2 is started i can see a static map, which will never get updated. In the TF-tree map and odom is in state Transform OK the rest shows No transform from ...
• On the remote PC i can echo lidar data, running ros2 topic echo /scan
• Both, the Raspberry and the remote PC, are connected to the same router via 5GHz-WLAN.
• Time is in sync on the Raspberry and the remote PC with the following command sudo ntpdate ntp.ubuntu.com

rviz2:

Best regards Tim

[roni-kreinin]

@timbo-r Is your Create3 connected to a 2.4GHz WiFi that is bridged with the 5GHz? Are you able to see topics such as /odom from your remote PC?

[timbo-r]

Hi @roni-kreinin, thank you for the quick response. Yes that is right: The Create3 is connected to the same WIFI (2.4GHz) as the Raspberry and the remote PC (5GHz). Also these are the only devices connected to the router, because it is a test environment dedicated to turtlebot scenarios. I can confirm that I can echo the odom data with ros2 topic echo /odom while controlling the robot via ros2 run teleop_twist_keyboard teleop_twist_keyboard on the remote PC.

ros2 topic echo /odom

---
header:
  stamp:
    sec: 1665083201
    nanosec: 62158851
  frame_id: odom
child_frame_id: base_link
pose:
  pose:
    position:
      x: -0.932745099067688
      y: 0.1608704924583435
      z: 0.0011600771686062217
    orientation:
      x: -0.006052435375750065
      y: 0.004871389828622341
      z: -0.9843089580535889
      w: 0.1762823462486267
  covariance:
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
[...]

[roni-kreinin]

Are you running this on a VM or old hardware? If you are getting the discarding message because the queue is full message repeatedly then it may be that slam_toolbox is not able to calculate fast enough. Can you try running slam_async.launch.py on your remote PC and see if you get the same issue?

[timbo-r]

Yes, you're right: the remote PC is running in a VM.

When running ros2 launch turtlebot4_navigation slam_async.launch.py, the error is the same as before:

[async_slam_toolbox_node-1] [INFO] [1665088015.191098923] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665088015.055 for reason 'discarding message because the queue is full'
[async_slam_toolbox_node-1] [INFO] [1665088015.330901187] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665088015.190 for reason 'discarding message because the queue is full'
[async_slam_toolbox_node-1] [INFO] [1665088015.484252599] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665088015.325 for reason 'discarding message because the queue is full'
[async_slam_toolbox_node-1] [INFO] [1665088015.584534679] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665088015.448 for reason 'discarding message because the queue is full'
[async_slam_toolbox_node-1] [INFO] [1665088015.719445599] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665088015.583 for reason 'discarding message because the queue is full'
[async_slam_toolbox_node-1] [INFO] [1665088015.854575600] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665088015.718 for reason 'discarding message because the queue is full'
[async_slam_toolbox_node-1] [INFO] [1665088015.994820038] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665088015.853 for reason 'discarding message because the queue is full'

And in the terminal running rviz with ros2 launch turtlebot4_viz view_robot.launch.py:

[rviz2-1] [INFO] [1665088016.815656080] [rviz2]: Message Filter dropping message: frame 'rplidar_link' at time 1665088015,448 for reason 'discarding message because the queue is full'
[rviz2-1] Warning: Invalid frame ID "wheel_drop_left" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "wheel_drop_right" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "wheel_drop_left" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "wheel_drop_right" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] [INFO] [1665088016.973487700] [rviz2]: Message Filter dropping message: frame 'rplidar_link' at time 1665088015,583 for reason 'discarding message because the queue is full'

Here is a screenshot of htop while running both commands:

[roni-kreinin]

From the slam_toolbox error messages it does seem that it is dropping every /scan message which is why the map doesn't update. Are you able to give the VM some more processing power?

[timbo-r]

I double the cores of the vm, but get the same errors ....

ros2 launch turtlebot4_navigation slam_async.launch.py

[async_slam_toolbox_node-1] [INFO] [1665089743.407484579] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665089743.265 for reason 'discarding message because the queue is full'
[async_slam_toolbox_node-1] [INFO] [1665089743.539295579] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665089743.401 for reason 'discarding message because the queue is full'
[async_slam_toolbox_node-1] [INFO] [1665089743.673941464] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665089743.536 for reason 'discarding message because the queue is full'
[async_slam_toolbox_node-1] [INFO] [1665089743.809411586] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665089743.671 for reason 'discarding message because the queue is full'
[async_slam_toolbox_node-1] [INFO] [1665089743.939112176] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665089743.806 for reason 'discarding message because the queue is full'
[async_slam_toolbox_node-1] [INFO] [1665089744.067080610] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665089743.929 for reason 'discarding message because the queue is full'
[async_slam_toolbox_node-1] [INFO] [1665089744.206619290] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665089744.064 for reason 'discarding message because the queue is full'
[async_slam_toolbox_node-1] [INFO] [1665089744.337312966] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665089744.199 for reason 'discarding message because the queue is full'
[async_slam_toolbox_node-1] [INFO] [1665089744.472001318] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665089744.334 for reason 'discarding message because the queue is full'
[async_slam_toolbox_node-1] [INFO] [1665089744.607985702] [slam_toolbox]: Message Filter dropping message: frame 'rplidar_link' at time 1665089744.469 for reason 'discarding message because the queue is full'

ros2 launch turtlebot4_viz view_robot.launch.py

[rviz2-1] Warning: Invalid frame ID "wheel_drop_right" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] [INFO] [1665089744.717440175] [rviz2]: Message Filter dropping message: frame 'rplidar_link' at time 1665089743,265 for reason 'discarding message because the queue is full'
[rviz2-1] Warning: Invalid frame ID "wheel_drop_left" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "wheel_drop_right" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] [INFO] [1665089744.868962972] [rviz2]: Message Filter dropping message: frame 'rplidar_link' at time 1665089743,401 for reason 'discarding message because the queue is full'
[rviz2-1] Warning: Invalid frame ID "wheel_drop_left" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "wheel_drop_right" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] [INFO] [1665089745.002023125] [rviz2]: Message Filter dropping message: frame 'rplidar_link' at time 1665089743,536 for reason 'discarding message because the queue is full'
[rviz2-1] Warning: Invalid frame ID "wheel_drop_left" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "wheel_drop_right" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] [INFO] [1665089745.131237899] [rviz2]: Message Filter dropping message: frame 'rplidar_link' at time 1665089743,671 for reason 'discarding message because the queue is full'
[rviz2-1] Warning: Invalid frame ID "wheel_drop_left" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "wheel_drop_right" passed to canTransform argument source_frame - frame does not exist
[rviz2-1]          at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.4/src/buffer_core.cpp
[rviz2-1] [INFO] [1665089745.233205773] [rviz2]: Message Filter dropping message: frame 'rplidar_link' at time 1665089743,806 for reason 'discarding message because the queue is full'

htop wile running both commands:

when running ros2 topic echo /scan i'm able to see the data of the lidar:

---
header:
  stamp:
    sec: 1665089807
    nanosec: 660888106
  frame_id: rplidar_link
angle_min: -3.1241393089294434
angle_max: 3.1415927410125732
angle_increment: 0.008714509196579456
time_increment: 0.00018494343385100365
scan_time: 0.13297432661056519
range_min: 0.15000000596046448
range_max: 12.0
ranges:
- 0.3479999899864197
- 0.34200000762939453
- 0.34299999475479126
- 0.34299999475479126
- 0.34200000762939453
- 0.33899998664855957
- 0.33899998664855957
- 0.3409999907016754
- 0.3409999907016754
- 0.34200000762939453
- 0.34200000762939453
- 0.34299999475479126
- 0.34299999475479126
- 0.3449999988079071
- 0.34599998593330383
[...]

i checked the network bandwidth with iperf3 and its nearly 60Mbits/sec (from the vm to the raspberry).

[Tauhid-Tanjim13]

I faced similar issue. I ran this command: ros2 launch turtlebot4_bringup rplidar.launch.py into RPI once or twice. It worked. Sometimes, I need to reset the whole network configuration. I am still trying understand the reason behind it.

[CycleMark]

Hi - Has there been any progress on this. I've seeing the same errors. If I run slam on the RPi or on my remote PC. Regardless of whether it's Async or not. The remote PC is more than powerful enough. Seems to be a bottleneck someplace.

Mark

[Basavaraj-PN]

Hi, I'm also getting same error as @timbo-r . create3 and RPi are on 2.4GHz interface Kindly help @roni-kreinin

[roni-kreinin]

iRobot has released G.4.3 firmware which has a "Restart ntpd" feature. I would recommend trying that to get the RPi and Create3 to sync.

[timbo-r]

I was able to test this scenario in a vm on another host with a befier cpu and the map was created without those errors mentioned above, but I'm still testing ... I will try the new firmware ... thank you @roni-kreinin ....

[Basavaraj-PN]

I tried restarting ntpd also, still no luck @roni-kreinin

[Basavaraj-PN]

Also getting this in Logs:user.notice ntpd: ntpd: timed out waiting for 192.168.186.1, reach 0x00, next query in 519s @roni-kreinin

[roni-kreinin]

@Basavaraj-PN Does the robot sync if you reboot the robot through the webserver? Application -> Reboot Robot

[Basavaraj-PN]

No it's not syncing.

[roni-kreinin]

@Basavaraj-PN Can you try the following:

• SSH into the RPi and edit the /etc/chrony/chrony.conf file.
• Comment out the server 192.168.186.2 presend 0 minpoll 0 maxpoll 0 iburst prefer trust line
• Save the file and restart the chrony service sudo systemctl restart chrony
• Restart ntpd on the Create 3

Let me know if that works.

[Basavaraj-PN]

@roni-kreinin, did exactly you said still not working.

[Basavaraj-PN]

Feb 28 04:48:11 iRobot-0470DA18552547709DEE29B254217E87 user.notice ntpd: ntpd: bad address '3.irobot.pool.ntp.org' Feb 28 04:48:13 iRobot-0470DA18552547709DEE29B254217E87 daemon.err network[1468]: [ERROR] (src/aos_net.c:449 errno:Address not available) route does not belong to main routing table Feb 28 04:48:13 iRobot-0470DA18552547709DEE29B254217E87 daemon.err network[1468]: [ERROR] (src/aos_net.c:249 errno:Address not available) aos_net_parse_routes failed Feb 28 04:48:13 iRobot-0470DA18552547709DEE29B254217E87 daemon.notice network[1468]: [INFO] (src/wifi_control.c:1542) have IP but have not read GW addr Feb 28 04:48:21 iRobot-0470DA18552547709DEE29B254217E87 daemon.notice network[1468]: [INFO] (src/wifi_control.c:1542) have IP but have not read GW addr Feb 28 04:48:21 iRobot-0470DA18552547709DEE29B254217E87 daemon.err network[1468]: [ERROR] (src/aos_net.c:449 errno:Address not available) route does not belong to main routing table Feb 28 04:48:21 iRobot-0470DA18552547709DEE29B254217E87 daemon.err network[1468]: [ERROR] (src/aos_net.c:249 errno:Address not available) aos_net_parse_routes failed Feb 28 04:48:21 iRobot-0470DA18552547709DEE29B254217E87 user.notice ntpd: ntpd: bad address '1.irobot.pool.ntp.org' Feb 28 04:48:22 iRobot-0470DA18552547709DEE29B254217E87 daemon.notice network[1468]: [INFO] (src/gateway_pinger.c:113) wifi is disconnected Feb 28 04:48:22 iRobot-0470DA18552547709DEE29B254217E87 daemon.notice network[1468]: [INFO] (src/gateway_pinger.c:219) no gateway to ping Feb 28 04:48:25 iRobot-0470DA18552547709DEE29B254217E87 daemon.notice network[1468]: [INFO] (src/wifi_control.c:1542) have IP but have not read GW addr Feb 28 04:48:25 iRobot-0470DA18552547709DEE29B254217E87 daemon.err network[1468]: [ERROR] (src/aos_net.c:449 errno:Address not available) route does not belong to main routing table Feb 28 04:48:26 iRobot-0470DA18552547709DEE29B254217E87 daemon.err network[1468]: [ERROR] (src/aos_net.c:249 errno:Address not available) aos_net_parse_routes failed Feb 28 04:48:26 iRobot-0470DA18552547709DEE29B254217E87 user.notice ntpd: ntpd: bad address '0.irobot.pool.ntp.org' Feb 28 04:48:26 iRobot-0470DA18552547709DEE29B254217E87 user.notice ntpd: ntpd: timed out waiting for 192.168.186.1, reach 0x00, next query in 32s Feb 28 04:48:42 iRobot-0470DA18552547709DEE29B254217E87 daemon.notice network[1468]: [INFO] (src/gateway_pinger.c:169) ready to ping using sudo arping -Iwlan0 -c1 192.168.0.1 -w 2 Feb 28 04:48:44 iRobot-0470DA18552547709DEE29B254217E87 user.notice create-platform: [INFO] [1677559724.164877297] [system_health]: CPU usage: max 100 [%] mean 100 [%] RAM usage: 29/59 [MB] Feb 28 04:48:44 iRobot-0470DA18552547709DEE29B254217E87 user.notice create-platform: Restarting create-platform Feb 28 04:48:46 iRobot-0470DA18552547709DEE29B254217E87 user.notice create-platform: [INFO] [1677559726.899081992] [create-platform]: Received termination signal 15 Feb 28 04:48:46 iRobot-0470DA18552547709DEE29B254217E87 user.notice create-platform: [INFO] [1677559726.899271031] [create-platform]: Shutting down nodes Feb 28 04:48:47 iRobot-0470DA18552547709DEE29B254217E87 user.notice create-platform: [INFO] [1677559727.017359043] [create-platform]: Shutdown mobility node

[Stanley-Sabeer]

Can i know is the issue is solved for anyone? I'm in an urgency. I can't sync lidar data. Please help. I'm getting the same message in terminal.

[sdobbelaere]

Actually I'm having the same issue. Would love to hear how others solved this.

[hilary-luo]

This issue is being closed due to inactivity. If you are still experiencing the issue, feel free to reopen this ticket when you are ready to continue the troubleshooting process.

[harsh0231]

facing the same issue [rviz2-1] [INFO] [1665068581.612268664] [rviz2]: Message Filter dropping message: frame 'rplidar_link' at time 1665068577,060 for reason 'discarding message because the queue is full help me to resolve