hello-robot / stretch_ros2

ROS 2 packages for the Stretch mobile manipulators from Hello Robot Inc.
https://docs.hello-robot.com/0.2/stretch-tutorials/ros2/
51 stars 19 forks source link

Feature/usb cam node #105

Closed hello-fazil closed 2 months ago

hello-fazil commented 3 months ago

This PR introduces a custom USB Cam node that uses OpenCV python to access USB camera feed from the given video port and publish it. The following parameters can be modified or set to use the node:

This PR also adds the following launch files:

hello-binit commented 3 months ago

Hi @hello-fazil, thanks for opening this PR. What is the frame rate I should expect from the beta teleop cameras? I'm seeing 1-3hz in my testing. With the stretch_camera_streams_check.py --usb_cam_port /dev/hello-gripper-camera tool, I'm seeing near real-time imagery, so I suspect there's a bottleneck at the ROS layer for some reason.

hello-fazil commented 2 months ago

Hi @hello-fazil, thanks for opening this PR. What is the frame rate I should expect from the beta teleop cameras? I'm seeing 1-3hz in my testing. With the stretch_camera_streams_check.py --usb_cam_port /dev/hello-gripper-camera tool, I'm seeing near real-time imagery, so I suspect there's a bottleneck at the ROS layer for some reason.

Hi @hello-binit , was the lag you mentioned observed while viewing the camera stream through the RQT image viewer or a Rviz camera view? If so, even I have been facing that lag while visualizing in rviz or rqt, which I suspect is to do something with the ROS layer and having a subscriber overhead. This thread suggests that likely the high network traffic within the NUC could cause this behavior.

But from my testing, functionally, I don't see this lag happening if I use a simple Python node to subscribe to the camera image topic and display it in real-time using cv2.imshow(), which produces real-time imagery similar to the tool stretch_camera_streams_check.py. I have added a print_debug param that will print the average stream latency, which could help us profile the lags between reading the frames from the USB port. I profiled these three cameras,

@hello-binit @hello-vinitha I would recommend you try out the below steps to verify if the camera is functionally streaming without lag and you are able to visualize it using a subscriber without lag: Start the gripper camera stream with print_debug, which will keep printing the average latency measured:

ros2 launch stretch_core beta_gripper_camera.launch.py print_debug:=True

Download this script ros2_simple_image_subscriber.py and run it to display the camera stream from the gripper:

python3 ros2_simple_image_subscriber.py --ros-args -p camera_topic:=/gripper_camera/image_raw

What latencies were reported, and did the imagery from the subscriber node feel real-time?

hello-vinitha commented 2 months ago

@hello-fazil I followed your instructions and did not see any noticeable lag with the gripper camera. The average latency was 123.7ms and average frequency was 8.08 Hz