iRobotEducation / create3_examples

Example nodes to drive the iRobot® Create® 3 Educational Robot
BSD 3-Clause "New" or "Revised" License
51 stars 12 forks source link

add create 3 republisher package #53

Closed alsora closed 2 months ago

slowrunner commented 3 months ago

This is exciting! Looking forward to trying it. (and seeing documentation on how to use the DDS unicast xml setup)

alsora commented 3 months ago

Hi @slowrunner, I just finished testing the package and with the proposed setup I'm able to have the laptop and RaspberryPi run ROS 2 entities (through the republisher) without those being discovered by the Create 3.

The documentation is in the provided README https://github.com/iRobotEducation/create3_examples/blob/asoragna/create3-republisher/create3_republisher/README.md

It would be great if you could give it a try with your rtabmap setup and provide any feedback. Hopefully this will address your issues withe robot running out of memory and going to 100% CPU.

slowrunner commented 3 months ago

Question: Do I continue to have the DDS server enabled on the Create3 and running on the RPi SBC?

I'm guessing no.

alsora commented 3 months ago

Correct. You should disable the discovery server, as the instructions use a different discovery mechanism

I'll add it to the readme

slowrunner commented 3 months ago

I think my first snag is the hard-coded param file. I don't like editing files inside cloned repositories (create3_examples), so I always copy the param files out, customize them, and then pass the customized file as a launch param.

I was overly confident and skipped the check it out before switching to the unicast segmentation, so I am in that "difficult to debug" situation you cautioned about. I'm not seeing topics with either the active or passive profile so I'm going to back out the unicast segmentation and verify as you instructed.

slowrunner commented 3 months ago

First launch of republisher was not a simple situation - republisher crashed, Plus other successes and failures

Crash Initial conditions:

more ../configs/super_client_configuration_file.xml <?xml version="1.0" encoding="UTF-8" ?>

SUPER_CLIENT
127.0.0.1
11811
- Wali_node had not received a /battery_state nor /dock_status so values are all default from a msg instantiation 
- Wali_node thinks {battery is low, not on dock, dock not visible} so repeatedly sends /rotate_angle in attempt to see dock
- Republisher not running

## Republisher Crashes When Launched with overlapping /rotate_angle?:
- Robot performed immediate rotate_angle on the dock
- republisher crashed

pi@WaLiPi5:~/wali_pi5/c3ws $ cmds/launch_republisher.sh pi@WaLiPi5:~/wali_pi5/c3ws $ [INFO] [launch]: All log files can be found below /home/pi/wali_pi5/c3ws/roslogs/2024-03-09-11-32-12-843544-WaLiPi5-241 [INFO] [launch]: Default logging verbosity is set to INFO RMW IMPLEMENTATION: rmw_fastrtps_cpp FAST-DDS CONFIG FILE: /home/pi/wali_pi5/configs/super_client_configuration_file.xml Using yaml file /home/pi/wali_pi5/c3ws/install/create3_republisher/share/create3_republisher/bringup/params.yaml [INFO] [create3_republisher-1]: process started with pid [242] [create3_republisher-1] [INFO] [1710001932.920080638] [create3_repub]: Creating republisher node with namespace '/' to interact with robot '/wali' [create3_republisher-1] [INFO] [1710001932.920551693] [create3_repub]: Found 8 entities for robot_publishers [create3_republisher-1] [INFO] [1710001932.920578749] [create3_repub]: Subscribing to topic '/wali/battery_state' and republishing it as '/battery_state' with type 'sensor_msgs/msg/BatteryState' [create3_republisher-1] [INFO] [1710001932.922939656] [create3_repub]: Subscribing to topic '/wali/dock_status' and republishing it as '/dock_status' with type 'irobot_create_msgs/msg/DockStatus' [create3_republisher-1] [INFO] [1710001932.925377026] [create3_repub]: Subscribing to topic '/wali/hazard_detection' and republishing it as '/hazard_detection' with type 'irobot_create_msgs/msg/HazardDetectionVector' [create3_republisher-1] [INFO] [1710001932.926354137] [create3_repub]: Subscribing to topic '/wali/ir_intensity' and republishing it as '/ir_intensity' with type 'irobot_create_msgs/msg/IrIntensityVector' [create3_republisher-1] [INFO] [1710001932.927305285] [create3_repub]: Subscribing to topic '/wali/ir_opcode' and republishing it as '/ir_opcode' with type 'irobot_create_msgs/msg/IrOpcode' [create3_republisher-1] [INFO] [1710001932.928167507] [create3_repub]: Subscribing to topic '/wali/odom' and republishing it as '/odom' with type 'nav_msgs/msg/Odometry' [create3_republisher-1] [INFO] [1710001932.929956526] [create3_repub]: Subscribing to topic '/wali/tf' and republishing it as '/tf' with type 'tf2_msgs/msg/TFMessage' [create3_republisher-1] [INFO] [1710001932.931459877] [create3_repub]: Subscribing to topic '/wali/tf_static' and republishing it as '/tf_static' with type 'tf2_msgs/msg/TFMessage' [create3_republisher-1] [INFO] [1710001932.932378248] [create3_repub]: Found 3 entities for robot_subscriptions [create3_republisher-1] [INFO] [1710001932.932415303] [create3_repub]: Subscribing to topic '/cmd_audio' and republishing it as '/wali/cmd_audio' with type 'irobot_create_msgs/msg/AudioNoteVector' [create3_republisher-1] [INFO] [1710001932.933416710] [create3_repub]: Subscribing to topic '/cmd_lightring' and republishing it as '/wali/cmd_lightring' with type 'irobot_create_msgs/msg/LightringLeds' [create3_republisher-1] [INFO] [1710001932.934423544] [create3_repub]: Subscribing to topic '/cmd_vel' and republishing it as '/wali/cmd_vel' with type 'geometry_msgs/msg/Twist' [create3_republisher-1] [INFO] [1710001932.935939544] [create3_repub]: Found 2 entities for robot_services [create3_republisher-1] [INFO] [1710001932.935978507] [create3_repub]: Remapping robot service '/wali/e_stop' as '/e_stop' with type 'irobot_create_msgs/srv/EStop' [create3_republisher-1] [INFO] [1710001932.937032636] [create3_repub]: Remapping robot service '/wali/robot_power' as '/robot_power' with type 'irobot_create_msgs/srv/RobotPower' [create3_republisher-1] [INFO] [1710001932.937910580] [create3_repub]: Found 9 entities for robot_actions [create3_republisher-1] [INFO] [1710001932.937947858] [create3_repub]: Remapping robot action '/wali/audio_note_sequence' as '/audio_note_sequence' with type 'irobot_create_msgs/action/AudioNoteSequence' [create3_republisher-1] [INFO] [1710001932.940851210] [create3_repub]: Remapping robot action '/wali/dock' as '/dock' with type 'irobot_create_msgs/action/Dock' [create3_republisher-1] [INFO] [1710001932.942970061] [create3_repub]: Remapping robot action '/wali/drive_arc' as '/drive_arc' with type 'irobot_create_msgs/action/DriveArc' [create3_republisher-1] [INFO] [1710001932.945214654] [create3_repub]: Remapping robot action '/wali/drive_distance' as '/drive_distance' with type 'irobot_create_msgs/action/DriveDistance' [create3_republisher-1] [INFO] [1710001932.947502802] [create3_repub]: Remapping robot action '/wali/led_animation' as '/led_animation' with type 'irobot_create_msgs/action/LedAnimation' [create3_republisher-1] [INFO] [1710001932.949973709] [create3_repub]: Remapping robot action '/wali/navigate_to_position' as '/navigate_to_position' with type 'irobot_create_msgs/action/NavigateToPosition' [create3_republisher-1] [INFO] [1710001932.952784598] [create3_repub]: Remapping robot action '/wali/rotate_angle' as '/rotate_angle' with type 'irobot_create_msgs/action/RotateAngle' [create3_republisher-1] [INFO] [1710001932.955470486] [create3_repub]: Remapping robot action '/wali/undock' as '/undock' with type 'irobot_create_msgs/action/Undock' [create3_republisher-1] [INFO] [1710001932.958324986] [create3_repub]: Remapping robot action '/wali/wall_follow' as '/wall_follow' with type 'irobot_create_msgs/action/WallFollow' [create3_republisher-1] [INFO] [1710001932.961342578] [create3_repub]: Ready to go! [create3_republisher-1] Received action request for /wali/rotate_angle [create3_republisher-1] Forwarding action request to /wali/rotate_angle [create3_republisher-1] Received action request for /wali/rotate_angle [create3_republisher-1] Forwarding action request to /wali/rotate_angle [create3_republisher-1] Action request /wali/rotate_angle received goal handle from the robot [create3_republisher-1] Action request /wali/rotate_angle received goal handle from the robot [ERROR] [create3_republisher-1]: process has died [pid 242, exit code -11, cmd '/home/pi/wali_pi5/c3ws/install/create3_republisher/lib/create3_republisher/create3_republisher --ros-args -r __node:=create3_repub -r __ns:=/ --params-file /home/pi/wali_pi5/c3ws/install/create3_republisher/share/create3_republisher/bringup/params.yaml --params-file /tmp/launch_params_6s7z6j31'].

[RepublisherCrash.txt](https://github.com/iRobotEducation/create3_examples/files/14547135/RepublisherCrash.txt)

# More tests (launch republisher before wali_node succeeds):
- I relaunched republisher (with Wali_node already running) and same exact crash. 
- I relaunched my ROS Docker container, which launches the republisher before starting wali_node:  
   - republisher remains alive
   - wali_node started up correctly  
     assessing the bot to be off the dock with sufficient battery  

pi@WaLiPi5:~/wali_pi5/c3ws $ ./start_docker_r2hdp_service.sh Starting Docker Service - 60s wait waiting for docker to start CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 5891b85c06be r2hdp "/ros_entrypoint.sh …" 10 seconds ago Up 9 seconds r2hdp pi@WaLiPi5:~/wali_pi5/c3ws $ docker logs r2hdp START_WALI.SH EXECUTING *** fastdds discovery -i 0 -l 192.168.186.3 -p 11811 -l 10.0.0.219 -p 11811 -l 127.0.0.1 -p 11811 &

Server is running

Participant Type: SERVER Security: NO Server ID: 0 Server GUID prefix: 44.53.00.5f.45.50.52.4f.53.49.4d.41 Server Addresses: UDPv4:[192.168.186.3]:11811 UDPv4:[10.0.0.219]:11811 UDPv4:[127.0.0.1]:11811 [INFO] [launch]: All log files can be found below /home/pi/.ros/log/2024-03-09-12-02-39-105301-WaLiPi5-61 [INFO] [launch]: Default logging verbosity is set to INFO FAST-DDS CONFIG FILE: /home/pi/wali_pi5/configs/super_client_configuration_file.xml Using yaml file /home/pi/wali_pi5/c3ws/install/create3_republisher/share/create3_republisher/bringup/params.yaml [INFO] [create3_republisher-1]: process started with pid [75] [create3_republisher-1] [INFO] [1710003759.181876627] [create3_repub]: Creating republisher node with namespace '/' to interact with robot '/wali' [create3_republisher-1] [INFO] [1710003759.182002275] [create3_repub]: Found 8 entities for robot_publishers [create3_republisher-1] [INFO] [1710003759.182019923] [create3_repub]: Subscribing to topic '/wali/battery_state' and republishing it as '/battery_state' with type 'sensor_msgs/msg/BatteryState' [create3_republisher-1] [INFO] [1710003759.184578071] [create3_repub]: Subscribing to topic '/wali/dock_status' and republishing it as '/dock_status' with type 'irobot_create_msgs/msg/DockStatus' [create3_republisher-1] [INFO] [1710003759.186265608] [create3_repub]: Subscribing to topic '/wali/hazard_detection' and republishing it as '/hazard_detection' with type 'irobot_create_msgs/msg/HazardDetectionVector' [create3_republisher-1] [INFO] [1710003759.187134237] [create3_repub]: Subscribing to topic '/wali/ir_intensity' and republishing it as '/ir_intensity' with type 'irobot_create_msgs/msg/IrIntensityVector' [create3_republisher-1] [INFO] [1710003759.187924293] [create3_repub]: Subscribing to topic '/wali/ir_opcode' and republishing it as '/ir_opcode' with type 'irobot_create_msgs/msg/IrOpcode' [create3_republisher-1] [INFO] [1710003759.188643237] [create3_repub]: Subscribing to topic '/wali/odom' and republishing it as '/odom' with type 'nav_msgs/msg/Odometry' [create3_republisher-1] [INFO] [1710003759.190192867] [create3_repub]: Subscribing to topic '/wali/tf' and republishing it as '/tf' with type 'tf2_msgs/msg/TFMessage' [create3_republisher-1] [INFO] [1710003759.191495589] [create3_repub]: Subscribing to topic '/wali/tf_static' and republishing it as '/tf_static' with type 'tf2_msgs/msg/TFMessage' [create3_republisher-1] [INFO] [1710003759.192264718] [create3_repub]: Found 3 entities for robot_subscriptions [create3_republisher-1] [INFO] [1710003759.192299255] [create3_repub]: Subscribing to topic '/cmd_audio' and republishing it as '/wali/cmd_audio' with type 'irobot_create_msgs/msg/AudioNoteVector' [create3_republisher-1] [INFO] [1710003759.193164218] [create3_repub]: Subscribing to topic '/cmd_lightring' and republishing it as '/wali/cmd_lightring' with type 'irobot_create_msgs/msg/LightringLeds' [create3_republisher-1] [INFO] [1710003759.194002922] [create3_repub]: Subscribing to topic '/cmd_vel' and republishing it as '/wali/cmd_vel' with type 'geometry_msgs/msg/Twist' [create3_republisher-1] [INFO] [1710003759.194926218] [create3_repub]: Found 2 entities for robot_services [create3_republisher-1] [INFO] [1710003759.194956422] [create3_repub]: Remapping robot service '/wali/e_stop' as '/e_stop' with type 'irobot_create_msgs/srv/EStop' [create3_republisher-1] [INFO] [1710003759.195689940] [create3_repub]: Remapping robot service '/wali/robot_power' as '/robot_power' with type 'irobot_create_msgs/srv/RobotPower' [create3_republisher-1] [INFO] [1710003759.196501977] [create3_repub]: Found 9 entities for robot_actions [create3_republisher-1] [INFO] [1710003759.196544532] [create3_repub]: Remapping robot action '/wali/audio_note_sequence' as '/audio_note_sequence' with type 'irobot_create_msgs/action/AudioNoteSequence' [create3_republisher-1] [INFO] [1710003759.199262273] [create3_repub]: Remapping robot action '/wali/dock' as '/dock' with type 'irobot_create_msgs/action/Dock' [create3_republisher-1] [INFO] [1710003759.201181013] [create3_repub]: Remapping robot action '/wali/drive_arc' as '/drive_arc' with type 'irobot_create_msgs/action/DriveArc' [create3_republisher-1] [INFO] [1710003759.203204698] [create3_repub]: Remapping robot action '/wali/drive_distance' as '/drive_distance' with type 'irobot_create_msgs/action/DriveDistance' [create3_republisher-1] [INFO] [1710003759.205450216] [create3_repub]: Remapping robot action '/wali/led_animation' as '/led_animation' with type 'irobot_create_msgs/action/LedAnimation' [create3_republisher-1] [INFO] [1710003759.207694790] [create3_repub]: Remapping robot action '/wali/navigate_to_position' as '/navigate_to_position' with type 'irobot_create_msgs/action/NavigateToPosition' [create3_republisher-1] [INFO] [1710003759.210086808] [create3_repub]: Remapping robot action '/wali/rotate_angle' as '/rotate_angle' with type 'irobot_create_msgs/action/RotateAngle' [create3_republisher-1] [INFO] [1710003759.212734512] [create3_repub]: Remapping robot action '/wali/undock' as '/undock' with type 'irobot_create_msgs/action/Undock' [create3_republisher-1] [INFO] [1710003759.215373919] [create3_repub]: Remapping robot action '/wali/wall_follow' as '/wall_follow' with type 'irobot_create_msgs/action/WallFollow' [create3_republisher-1] [INFO] [1710003759.218138826] [create3_repub]: Ready to go!

Start F710 game controller node ros2 launch teleop_twist_joy teleop-launch.py joy_config:="F710"

Start odometer node ros2 run wali odometer &

Start wali.say_node ros2 run wali say_node & [INFO] [launch]: All log files can be found below /home/pi/.ros/log/2024-03-09-12-02-44-305289-WaLiPi5-91 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [joy_node-1]: process started with pid [98] [INFO] [teleop_node-2]: process started with pid [100] [teleop_node-2] [INFO] [1710003764.550111369] [TeleopTwistJoy]: Teleop enable button 4. [teleop_node-2] [INFO] [1710003764.550185073] [TeleopTwistJoy]: Turbo on button 5. [teleop_node-2] [INFO] [1710003764.550191036] [TeleopTwistJoy]: Linear axis x on 1 at scale 0.150000. [teleop_node-2] [INFO] [1710003764.550200110] [TeleopTwistJoy]: Turbo for linear axis x is scale 0.300000. [teleop_node-2] [INFO] [1710003764.550205665] [TeleopTwistJoy]: Angular axis yaw on 0 at scale 0.250000. [teleop_node-2] [INFO] [1710003764.550209925] [TeleopTwistJoy]: Turbo for angular axis yaw is scale 0.500000. [joy_node-1] [INFO] [1710003764.682226258] [joy_node]: Opened joystick: Logitech Gamepad F710. deadzone: 0.300000 WaLI node started - Undock:99% Rotate:18% Dock:15% pi@WaLiPi5:~/wali_pi5/c3ws $


- Tested /rotate_angle - success
- Tested /dock - success
- Tested /undock - success
- Tested /odom - success
- Tested /battery_state - success
- Tested /dock_status - success
- Tested /hazard_detection - success
- Tested /ir_intensity - success
- Tested /ir_opcode - success (sees virtual wall)
- Tested /robot_power - success

## Tested /cmd_vel - success BUT backup reflex fired - ignored full safety override 
(Tested after restarting the Create3 application,  and after a full cold start of Create3 and SBC)

motion_control: ros__parameters:

safety_override options are

# "none" - standard safety profile, robot cannot backup more than an inch because of lack of cliff protection in rear, max speed 0.306m/s
# "backup_only" - allow backup without cliff safety, but keep cliff safety forward and max speed at 0.306m/s
# "full" - no cliff safety, robot will ignore cliffs and set max speed to 0.46m/s
safety_override: "full"

robot_state: ros__parameters:

publish_hazard_msgs: default true, set false to noticeably reduce CPU and RAM useage

publish_hazard_msgs: false
# publish_odom_tfs: default true, if not needed will noticeably reduce CPU and RAM use
publish_odom_tfs: false 
# raw_kinematics_min_pub_period_ms: default 25 , -1 disables /imu, /mouse, /wheel_status, /wheel_ticks, /wheel_vels (-1 recommended)
raw_kinematics_min_pub_period_ms: -1
# hazards_pub_fixed_period_ms: default -1 disables fixed period hazard publication, pubs only when change noted, 1: publish as soon as possible, 2+ period for constant publishing 

Mar 9 17:35:02 Create3-WaLi user.notice create-platform: [INFO] [1710005702.785172370] [wali.system_health]: CPU usage: max 79 [%] mean 67 [%] RAM usage: 32/59 [MB] Mar 9 17:35:12 Create3-WaLi user.notice create-platform: [WARN] [1710005712.577603159] [wali.motion_control]: Reached backup limit! Stop Driving robot backward or disable from safety_override parameter! Mar 9 17:35:14 Create3-WaLi user.notice create-platform: [WARN] [1710005714.277378692] [wali.motion_control]: Reached backup limit! Stop Driving robot backward or disable from safety_override parameter! Mar 9 17:35:15 Create3-WaLi user.notice create-platform: [WARN] [1710005715.127595960] [wali.motion_control]: Reached backup limit! Stop Driving robot backward or disable from safety_override parameter! Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [WARN] [1710005723.727822776] [wali.motion_control]: Reached backup limit! Stop Driving robot backward or disable from safety_override parameter! Mar 9 17:35:23 Create3-WaLi user.notice create-platform: 99 id: 25 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [WAR] Packet handler running too long exec_time: 54 id: 25 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:3085,-424 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:-300,-300 meas_vel:-193,-189 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:1955,117 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:-34,30 meas_vel:0,0 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:36,36 meas_vel:0,0 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:26,-26 meas_vel:0,0 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:3940,4056 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:-27,27 meas_vel:-2,0 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:3245,4749 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:31,-30 meas_vel:0,0 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:32,-29 meas_vel:0,0 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:4370,4169 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:-30,-30 meas_vel:-1,0 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:-26,-26 meas_vel:0,0 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:-26,-26 meas_vel:0,0 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:26,26 meas_vel:0,0 Mar 9 17:35:23 Create3-WaLi user.notice create-platform: [DBG] [SE]: !wheels_stopped - odom:4352,4152 Mar 9 17:35:39 Create3-WaLi user.notice create-platform: [DBG] [SE]: !velocity_stopped - cmd_vel:[INFO] [1710005739.535759605] [wali.mobility_monitor]: Received mobility notification: ROBOT_NOTIFY_BACKUP_REFUSED (88) with arg 0 Mar 9 17:36:02 Create3-WaLi user.notice create-platform: [INFO] [1710005762.779060162] [wali.system_health]: CPU usage: max 91 [%] mean 73 [%] RAM usage: 32/59 [MB] Mar 9 17:37:02 Create3-WaLi user.notice create-platform: [INFO] [1710005822.782609324] [wali.system_health]: CPU usage: max 82 [%] mean 70 [%] RAM usage: 32/59 [MB]



## Tested /reset_pose - (not available in republish params.yaml) - Failed

## So now I need to redo the unicast segmentation and try again
alsora commented 3 months ago

Wow, thanks a lot for all the details.

slowrunner commented 3 months ago

I figured out the unicast not working issue - I was exporting the Discovery Server in my .bashrc file. Now have active and passive environments with republisher bridging them. (and full safety override working .. that remains a mystery but working so I can't recreate the issue)

Have to debug something weird with the rtabmap launch now. Node is launching but isn't starting all the components.

Going to sleep on that one.

slowrunner commented 3 months ago

Question: Does the 4096 fastdds buffer size limit in the passive profile allow 1MB image topics to be published and subscribed to?

 <!-- Reduce socket buffer size -->
            <sendBufferSize>4096</sendBufferSize>
            <receiveBufferSize>4096</receiveBufferSize>
            <!-- Reduce max message size, otherwise the participant creation fails -->
            <maxMessageSize>4096</maxMessageSize>

Does the republisher need its own passive profile, and camera and RTABmap nodes use a different passive profile without a limit?

slowrunner commented 3 months ago

QUESTION: Where's Wali? Where's /scan

Issues with dual unicast profile setup (passive/user node segment):

pi@WaLiPi5:~/wali_pi5/c3ws $ ros2 topic echo /scan WARNING: topic [/scan] does not appear to be published yet Could not determine the type for the passed topic

pi@WaLiPi5:~/wali_pi5/c3ws $ ros2 topic list /battery_state /cmd_audio /cmd_lightring /cmd_vel /dock_status /hazard_detection /ir_intensity /ir_opcode /joy /joy/set_feedback /odom /parameter_events /rosout /stop_status <<---- /scan is missing /tf /tf_static /wali/battery_state /wali/cmd_audio /wali/cmd_lightring /wali/cmd_vel /wali/dock_status /wali/hazard_detection /wali/ir_intensity /wali/ir_opcode /wali/odom /wali/stop_status /wali/tf /wali/tf_static

pi@WaLiPi5:~/wali_pi5/c3ws $ ros2 node list /create3_repub /joy_node /teleop_twist_joy_node


### CONFIGURATION

Create3:
- namespace /wali
[Override_RMW_XML_Profile.txt](https://github.com/iRobotEducation/create3_examples/files/14559564/Override_RMW_XML_Profile.txt)
- not checked: Enable FastDDS Discovery Server (and addr:port empty)

SBC (RPi5/Docker):
- Nodes running:
  - "active unicast profile":  create3_repub
            <initialPeersList>
                <locator>
                    <udpv4>
                        <address>127.0.0.1</address>
                        <address>192.168.186.2</address>
                    </udpv4>
                </locator>
            </initialPeersList>
  - "passive unicast profile":  
    - wali_node:  ```ros2 run wali wali_node```
    - odometer node:  ```ros2 run wali odometer &```
    - say node:  ```ros2 run wali say &```
    - ir2scan: ```ros2 run wali ir2scan```     <-- subscribes/sees repub'd /ir_intensity, 
                                                                                publishes /scan but cannot be echo'd
    - 
    - teleop_twist_joy: ```ros2 launch teleop_twist_joy teleop-launch.py joy_config:='F710' & 
    - joy_node:  (started by teleop-launch.py launch file)
            <initialPeersList>
                <locator>
                    <udpv4>
                        <address>127.0.0.1</address>
                    </udpv4>
                </locator>
            </initialPeersList>

ifconfig output:

pi@WaLiPi5:~/wali_pi5/c3ws $ ifconfig docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255 ether 02:42:d4:05:98:2c txqueuelen 0 (Ethernet) RX packets 207000 bytes 10725956 (10.7 MB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 541112 bytes 814928164 (814.9 MB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eth0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 ether d8:3a:dd:b6:36:a1 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 106

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 loop txqueuelen 1000 (Local Loopback) RX packets 6324674 bytes 2737691456 (2.7 GB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 6324674 bytes 2737691456 (2.7 GB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.186.3 netmask 255.255.0.0 broadcast 192.168.255.255 ether 0e:a2:25:c2:c5:6a txqueuelen 1000 (Ethernet) RX packets 9771757 bytes 4572207104 (4.5 GB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 305663 bytes 91753699 (91.7 MB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

usb0.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 ether 0e:a2:25:c2:c5:6a txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1603 bytes 548226 (548.2 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.0.219 netmask 255.255.255.0 broadcast 10.0.0.255 ether d8:3a:dd:b6:36:a2 txqueuelen 1000 (Ethernet) RX packets 11252848 bytes 6078606619 (6.0 GB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1046386 bytes 344754641 (344.7 MB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

[fastdds-passive-unicast.xml.txt](https://github.com/iRobotEducation/create3_examples/files/14559477/fastdds-passive-unicast.xml.txt)

[fastdds-active-unicast.xml.txt](https://github.com/iRobotEducation/create3_examples/files/14559478/fastdds-active-unicast.xml.txt)

After running ```source ss_passive.sh```

pi@WaLiPi5:~/wali_pi5/c3ws $ more ss_passive.sh

!/bin/bash

if [ -f /opt/ros/galactic/setup.bash ]; then

source /opt/ros/galactic/setup.bash

fi

if [ -f /opt/ros/humble/setup.bash ]; then source /opt/ros/humble/setup.bash fi

if [ -f /opt/ros/humble/local_setup.bash ]; then if [ -f ~/wali_pi5/c3ws/install/setup.bash ]; then source ~/wali_pi5/c3ws/install/setup.bash echo -e "sourced c3ws install setup.bash" fi

if [ -f ~/wali_pi5/dai_ws/install/setup.bash ]; then source ~/wali_pi5/dai_ws/install/setup.bash echo -e "sourced dai_ws install setup.bash" fi fi

export RMW_IMPLEMENTATION=rmw_fastrtps_cpp

Set up for republisher

export FASTRTPS_DEFAULT_PROFILES_FILE=~/wali_pi5/configs/fastdds-passive-unicast.xml ros2 daemon stop

ros2 topic list -----------EOF---------

pi@WaLiPi5:~/wali_pi5/c3ws $ printenv | grep DISC <<-- No Discovery Server export

pi@WaLiPi5:~/wali_pi5/c3ws $ printenv | grep FILE FASTRTPS_DEFAULT_PROFILES_FILE=/home/pi/wali_pi5/configs/fastdds-passive-unicast.xml

pi@WaLiPi5:~/wali_pi5/c3ws $ printenv | grep ROS ROS_VERSION=2 ROS_PYTHON_VERSION=3 ROS_LOG_DIR=/home/pi/wali_pi5/c3ws/roslogs ROS_LOCALHOST_ONLY=0 ROS_DISTRO=humble


This is the script Docker ROS 2 runs when it is started - launches repub and all Create3-Wali nodes:

!/bin/bash

This script is run when Docker container r2hdp starts

echo -e "START_WALI.SH EXECUTING" cd /home/pi/wali_pi5/c3ws source install/setup.bash

Launch the republisher with the active profile

export FASTRTPS_DEFAULT_PROFILES_FILE=~/wali_pi5/configs/fastdds-active-unicast.xml ros2 daemon stop ros2 launch create3_republisher create3_republisher_launch.py robot_ns:=/wali republisher_ns:=/ &

Launch all apps with the passive profile

export FASTRTPS_DEFAULT_PROFILES_FILE=~/wali_pi5/configs/fastdds-passive-unicast.xml ros2 daemon stop

echo -e "\n Start F710 game controller node" echo ' ros2 launch teleop_twist_joy teleop-launch.py joy_config:="F710" ' ros2 launch teleop_twist_joy teleop-launch.py joy_config:="F710" &

echo -e "\n Start odometer node" echo ' ros2 run wali odometer & ' ros2 run wali odometer &

echo -e "\n Start wali.say_node" echo ' ros2 run wali say_node &' ros2 run wali say_node &

echo -e "\n **** STARTING ir2scan " echo -e "executing: ros2 run wali ir2scan &" ros2 run wali ir2scan &

MUST START IN FOREGROUND or docker will exit!

ros2 run wali wali_node

Use ./attach_to_docker_r2hdp.sh to connect to this terminal

Use ./term_to_r2hdp.sh to open a new terminal session

in the ROS 2 Humble Desktop Plus container

alsora commented 3 months ago

You need all your processes to use unicast profiles.

You mentioned that you are not able to echo topics or to send service requests. You need to select a unicast profile also in the terminal where you are doing these things.

If you are doing it from the raspberry pi, you should use the passive profile. If you are doing it from another device (e.g. your laptop) you will need to create a dedicated profile as described in the optional point 5 of the instructions in the readme

slowrunner commented 3 months ago

You need all your processes to use unicast profiles.

Is the case

You mentioned that you are not able to echo topics or to send service requests. You need to select a unicast profile also in the terminal where you are doing these things.

If you are doing it from the raspberry pi, you should use the passive profile.

Yes, this Is the case: 1) Use passive profile:

export FASTRTPS_DEFAULT_PROFILES_FILE=~/wali_pi5/configs/fastdds-passive-unicast.xml
ros2 daemon stop

2) run ros2 node list - does not show all active nodes in passive unicast segment:

pi@WaLiPi5:~/wali_pi5/c3ws $ ros2 node list
/create3_repub
/odometer
/teleop_twist_joy_node
                                                   <<-- missing wali_node, say node, ir2scan node
                                                            interesting - joy_node used to show, doesn't this time, odometer shows hasn't before

3) execute ros2 topic list

/battery_state
/cmd_audio
/cmd_lightring
/cmd_vel
/dock_status
/hazard_detection
/ir_intensity
/ir_opcode
/joy
/odom
/parameter_events
/rosout
/stop_status
/tf
/tf_static
/wali/battery_state
/wali/cmd_audio
/wali/cmd_lightring
/wali/cmd_vel
/wali/dock_status
/wali/hazard_detection
/wali/ir_intensity
/wali/ir_opcode
/wali/odom
/wali/stop_status
/wali/tf
/wali/tf_static
                                    <<--- missing /scan topic emitted from ir2scan node

pi@WaLiPi5:~/wali_pi5/c3ws $ ps -ef | grep ir2scan
pi            85      44  0 09:25 pts/0    00:00:00 /usr/bin/python3 /opt/ros/humble/bin/ros2 run wali ir2scan
pi            93      85  7 09:25 pts/0    00:14:33 /usr/bin/python3 /home/pi/wali_pi5/c3ws/install/wali/lib/wali/ir2scan
pi           741     343  0 12:50 pts/2    00:00:00 grep --color=auto ir2scan

If you are doing it from another device (e.g. your laptop) you will need to create a dedicated profile as described in the optional point 5 of the instructions in the readme

I tried using a "desktop" profile thinking perhaps I should treat the Docker Container as another device - no cigars:

pi@WaLiPi5:~/wali_pi5/c3ws $ more ../configs/fastdds-desktop-unicast.xml 
<?xml version="1.0" encoding="UTF-8" ?>
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
    <transport_descriptors>
        <transport_descriptor>
            <transport_id>udp_transport</transport_id>
            <type>UDPv4</type>
            <!-- Reduce socket buffer size -->
            <sendBufferSize>4096</sendBufferSize>
            <receiveBufferSize>4096</receiveBufferSize>
            <!-- Reduce max message size, otherwise the participant creation fails -->
            <maxMessageSize>4096</maxMessageSize>
        </transport_descriptor>
    </transport_descriptors>

    <participant profile_name="unicast_discovery" is_default_profile="true">
        <rtps>
            <userTransports>
                <transport_id>udp_transport</transport_id>
            </userTransports>
            <useBuiltinTransports>false</useBuiltinTransports>
            <builtin>
                <metatrafficUnicastLocatorList>
                    <locator/>
                </metatrafficUnicastLocatorList>
                <initialPeersList>
                    <locator>
                        <udpv4>
                            <address>127.0.0.1</address>
                            <address>10.0.0.219</address>
                        </udpv4>
                    </locator>
                </initialPeersList>
            </builtin>
        </rtps>
    </participant>
</profiles>

pi@WaLiPi5:~/wali_pi5/c3ws $ source ss_desktop.sh 
sourced c3ws install setup.bash
sourced dai_ws install setup.bash
The daemon has been stopped
/battery_state
/cmd_audio
/cmd_lightring
/cmd_vel
/dock_status
/hazard_detection
/ir_intensity
/ir_opcode
/joy
/odom
/parameter_events
/rosout                                     <<--- /scan missing
/stop_status
/tf
/tf_static
/wali/battery_state
/wali/cmd_audio
/wali/cmd_lightring
/wali/cmd_vel
/wali/dock_status
/wali/hazard_detection
/wali/ir_intensity
/wali/ir_opcode
/wali/odom
/wali/stop_status
/wali/tf
/wali/tf_static

pi@WaLiPi5:~/wali_pi5/c3ws $ ros2 node list
/create3_repub
/odometer                        <<-- missing wali_node, ir2scan node,  say node
/teleop_twist_joy_node
slowrunner commented 3 months ago

If you would like to try my ir2scan node in your environment:

https://github.com/slowrunner/wali_pi5/blob/main/c3ws/src/wali/wali/ir2scan.py

If uncomment DEBUG=True to see that it is receiving /ir_intensity from republisher, and publishing /scan topics

ir_dist.py must be in the same folder - imported by ir2scan.py

https://github.com/slowrunner/wali_pi5/blob/main/c3ws/src/wali/wali/ir_dist.py

slowrunner commented 3 months ago

I have recreated my Create3-wali on a Raspberry Pi4 running ROS 2 Humble / Ubuntu 22.04 native (Republisher consumes 15% of Pi4 with IMU and tf turned off)

SUCCESSES:

[INFO] [1710263462.735952136] [say]: Say request:"hello from the say_node" response: wali_interfaces.srv.Say_Response(spoken=True)


ISSUES (same as Pi5/Docker):
- node ir2scan is publishing /scan but 
  - ```ros2 topic echo /scan``` does not show any published msgs
  - ```ros2 topic hz /scan``` 

WARNING: topic [/scan] does not appear to be published yet


- node list and topic list do not list all  (after a daemon stop/start or with --no-daemon)

ANALYSIS:
- /scan publish not succeeding is real unicast configuration issue - not Docker related
- node and topic list issue is still real unicast configuration issue - not Docker related

## Any clue?  I'm blocked - don't have any further ideas to try on my end.
alsora commented 3 months ago

Thank you for trying on the pi4.

I'm confident that the issues you see (ros2 topic echo not showing data) are caused by a misconfiguration of the xml profiles. In particular, the xml profiles involved are the one used by scan publisher and the one used in the terminal where you run the command.

I assume that both components (the scan publisher and the terminal running ros2 topic echo) are running on the raspberry pi. They should both use the passive unicast xml profile: create3_republisher/dds-config/fastdds-passive-unicast.xml.

I would also suggest to try to remove the lines limiting the maximum packet size from all xml profiles except the one running on create 3.

Is this a problem also with other topics or only with the scan one?

I'm sorry if you already tried some of these things, but I'm currently without access to a laptop and writing from my phone. I'll be able to get a better understanding of the problem next week once I'm back.

slowrunner commented 3 months ago

@alsora I am very grateful for your efforts to help me to this point. I am sorry to report I have abandoned this dream.