Drive unit malfunctioning

alexswerner commented 2 months ago

Robot Model

Turtlebot4 Standard

ROS distro

Humble

Networking Configuration

Discovery Server

OS

Ubuntu 22.04

Built from source or installed?

Built from Source

Package version

Base: H2.6.

Expected behaviour

The robot should move forward or backward in a straight line when commanded to do so.

Actual behaviour

When commanding the robot to drive forward, the robot is spinning on the spot, alternating between clock-wise and counter-clock-wise every 500ms. The same happens when command backwards.

Error messages

Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [WARN] [1726843255.423229455] [motion_control]: Reached backup limit! Stop Driving robot backward or disable
from safety_override parameter!
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: 221,-221 meas_vel:191,-184
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:473,-1182
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:221,-221 meas_vel:192,-159
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [WAR] Packet handler running too long exec_time: 21 id: 25
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:1564,-2534
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:0,0 meas_vel:-1,0
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:1552,-2524
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:-221,221 meas_vel:-171,196
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:1907,-3046
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:221,-221 meas_vel:211,-191
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:3457,-4595
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:221,-221 meas_vel:180,-185
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:12,-12 meas_vel:0,0
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:4791,-5932
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:-221,221 meas_vel:-168,181
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:6,-6 meas_vel:0,0
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:5025,-6164
Sep 20 14:40:55 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:-221,221 meas_vel:-171,197
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:4766,-5905
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:-221,221 meas_vel:-148,170
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:3,-3 meas_vel:0,0
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:7,-7 meas_vel:0,0
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:4492,-5628
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:0,0 meas_vel:-4,0
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:14,-14 meas_vel:0,0
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:4635,-5769
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:221,-221 meas_vel:213,-179
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:3,-3 meas_vel:0,0
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:5007,-6141
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:30,-30 meas_vel:0,0
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:6,-6 meas_vel:0,0
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:11,-11 meas_vel:0,0
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:5373,-6509
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:-221,221 meas_vel:-160,166
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:11,-11 meas_vel:0,0
Sep 20 14:41:15 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:16,-16 meas_vel:0,0
Sep 20 14:41:16 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:6,-[INFO] [1726843276.166011929] [system_health]: CPU usage: max 84
[%] mean 78 [%] RAM usage: 31/59 [MB]
Sep 20 14:41:22 uwbot-15-base user.notice ntpd: ntpd: timed out waiting for 192.168.186.1, reach 0x00, next query in 541s
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: 6 meas_vel:0,0
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:5395,-6532
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:-221,221 meas_vel:-157,177
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:4341,-5481
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:-221,221 meas_vel:-185,173
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:-17,-17 meas_vel:0,0
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:4223,-5363
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:221,-221 meas_vel:187,-177
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:11,-11 meas_vel:0,0
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:2,-2 meas_vel:0,0
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:2,-2 meas_vel:0,0
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:8,-8 meas_vel:0,0
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:4991,-6131
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:221,-221 meas_vel:181,-180
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:15,-15 meas_vel:0,0
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:6,-6 meas_vel:0,0
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:7,-7 meas_vel:0,0
Sep 20 14:41:48 uwbot-15-base user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:3,-3 meas_vel:0,0

ubuntu@uwbot-15:~$ ros2 topic echo --once /battery_state
header:
stamp:
sec: 1726853099
nanosec: 512095865
frame_id: ''
voltage: 15.463000297546387
temperature: 37.349998474121094
current: -0.8700000047683716
charge: 0.652999997138977
capacity: 1.0
design_capacity: 1.0
percentage: 0.6499999761581421
power_supply_status: 0
power_supply_health: 0
power_supply_technology: 0
present: true
cell_voltage: []
cell_temperature: []
location: ''
serial_number: ''
---

To Reproduce

Launch console application for keyboard teleoperation. Command the robot to move forward and backwards.

Other notes

Performed factory reset of the base, but this did not change the behavior.

alexswerner commented 2 months ago

We replaced the drive units of the base which did not fix the issue.

alexswerner commented 2 months ago

It looks like a bumper sensor is being triggered when starting to drive.

alexswerner commented 2 months ago

After cleaning the bumper inside I'm still getting:

header:
  stamp:
    sec: 0
    nanosec: 0
  frame_id: base_link
detections:
- header:
    stamp:
      sec: 1727197777
      nanosec: 639798548
    frame_id: base_link
  type: 0
---
header:
  stamp:
    sec: 0
    nanosec: 0
  frame_id: base_link
detections:
- header:
    stamp:
      sec: 1727197777
      nanosec: 991799204
    frame_id: base_link
  type: 0
---
header:
  stamp:
    sec: 0
    nanosec: 0
  frame_id: base_link
detections: []
---

And after this the robot still stops/turns around.

alexswerner commented 2 months ago

And /ir_intensity is showing:

---
header:
  stamp:
    sec: 1727197947
    nanosec: 740484766
  frame_id: base_link
readings:
- header:
    stamp:
      sec: 1727197947
      nanosec: 740484766
    frame_id: ir_intensity_side_left
  value: 0
- header:
    stamp:
      sec: 1727197947
      nanosec: 740484766
    frame_id: ir_intensity_left
  value: 3
- header:
    stamp:
      sec: 1727197947
      nanosec: 740484766
    frame_id: ir_intensity_front_left
  value: 0
- header:
    stamp:
      sec: 1727197947
      nanosec: 740484766
    frame_id: ir_intensity_front_center_left
  value: 1
- header:
    stamp:
      sec: 1727197947
      nanosec: 740484766
    frame_id: ir_intensity_front_center_right
  value: 4
- header:
    stamp:
      sec: 1727197947
      nanosec: 740484766
    frame_id: ir_intensity_front_right
  value: 0
- header:
    stamp:
      sec: 1727197947
      nanosec: 740484766
    frame_id: ir_intensity_right
  value: 1
---

Where all values are flickering from 0-4.

alexswerner commented 2 months ago

Setting reflexes_enabled: false in the motion_control yaml configuration does fix the behavior. My guess is that the bumper electronics have an issue (since it affects all bumpers). Temporarily this is acceptable but some functionality of the base is apparently still impacted - automatic docking does not work.

RustyCPR commented 2 months ago

Hi there Alexander for creating this ticket.
I'm sorry to see your TB4 is experiencing this issue. Could you please replicate your concern and then please share the Create3 logs?
Big thanks, Rusty

alexswerner commented 1 month ago

base.log @RustyCPR Here is the log. I rebooted the robot and let it start completely. Then backed it off the base and drove it onto the base (manually). This is with reflexes_enabled: false, so I'm not sure if the issue is visible. If needed I can provide another log with reflexed_enabled: true. Just let me know.

smatarCPR commented 1 month ago

Hello @alexswerner , Thank you for providing the logs. Would it be possible to provide logs with reflexed_enabled: true and if possible, the strange driving behaviour replicated.

Looking forward to hearing from you soon. Best Regards, Saif

alexswerner commented 1 month ago

base.log Here is the log with reflexed_enabled: true. It immediately displayed the strange behavior. In this session I rebooted the base while the robot was on the dock. Then drove it backwards off the base. This resulted in the robot spinning, like it was trying to avoid an obstacle.

smatarCPR commented 1 month ago

Hello @alexswerner Thank you for sending these logs. We'll begin our analysis and let you know what we find. Looking forward to connecting soon. Best Regards, Saif

smatarCPR commented 3 weeks ago

Hello @alexswerner , Apologies for the late reply on this. Reviewing the log details it seems that the drive issue may be due to a firmware issue on the Create 3. Would it be possible to try and:

Update the Create3 to the latest firmware version H.2.6, if this version is already loaded, reflash it by downloading the firmware file from here - https://iroboteducation.github.io/create3_docs/releases/overview/ and uploading it to the Create3
Following the firmware update perform a factory reset using the About tab in the webserver - https://iroboteducation.github.io/create3_docs/webserver/about/

Following the above process let us know if the Turtlebot 4 unit still exhibits the erratic driving behaviour.

Looking forward to hearing from you soon. Best Regards, Saif

smatarCPR commented 1 week ago

Hello @alexswerner , It has been quite some time since I last heard from you. Do you have any updates to share? Looking forward to hearing from you soon. Best Regards, Saif

turtlebot / turtlebot4