search

hello-robot / stretch_web_teleop

Remote web teleoperation for the Stretch mobile manipulators from Hello Robot Inc.
Other
15 stars 1 forks source link

configure_video_streams.py dying when depth visualization turned on #27

Closed hello-binit closed 5 months ago

hello-binit commented 9 months ago

I turned on Depth Sensing visualization in the interface, and a few moments later, I see this error and the video streams freeze:

[configure_video_streams.py-6] Publishing reconfigured video stream
[configure_video_streams.py-6] depth
[configure_video_streams.py-6] depth
[configure_video_streams.py-6] depth
[configure_video_streams.py-6] depth
[configure_video_streams.py-6] depth
[configure_video_streams.py-6] Traceback (most recent call last):
[configure_video_streams.py-6]   File "/home/hello-robot/ament_ws/install/stretch_web_teleop/lib/stretch_web_teleop/configure_video_streams.py", line 318, in <module>
[configure_video_streams.py-6]     rclpy.spin(node)
[configure_video_streams.py-6]   File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/__init__.py", line 222, in spin
[configure_video_streams.py-6]     executor.spin_once()
[configure_video_streams.py-6]   File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 739, in spin_once
[configure_video_streams.py-6]     self._spin_once_impl(timeout_sec)
[configure_video_streams.py-6]   File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 736, in _spin_once_impl
[configure_video_streams.py-6]     raise handler.exception()
[configure_video_streams.py-6]   File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/task.py", line 239, in __call__
[configure_video_streams.py-6]     self._handler.send(None)
[configure_video_streams.py-6]   File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 437, in handler
[configure_video_streams.py-6]     await call_coroutine(entity, arg)
[configure_video_streams.py-6]   File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 362, in _execute_subscription
[configure_video_streams.py-6]     await await_or_execute(sub.callback, msg)
[configure_video_streams.py-6]   File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 107, in await_or_execute
[configure_video_streams.py-6]     return callback(*args)
[configure_video_streams.py-6]   File "/opt/ros/humble/local/lib/python3.10/dist-packages/message_filters/__init__.py", line 83, in callback
[configure_video_streams.py-6]     self.signalMessage(msg)
[configure_video_streams.py-6]   File "/opt/ros/humble/local/lib/python3.10/dist-packages/message_filters/__init__.py", line 64, in signalMessage
[configure_video_streams.py-6]     cb(*(msg + args))
[configure_video_streams.py-6]   File "/opt/ros/humble/local/lib/python3.10/dist-packages/message_filters/__init__.py", line 313, in add
[configure_video_streams.py-6]     self.signalMessage(*msgs)
[configure_video_streams.py-6]   File "/opt/ros/humble/local/lib/python3.10/dist-packages/message_filters/__init__.py", line 64, in signalMessage
[configure_video_streams.py-6]     cb(*(msg + args))
[configure_video_streams.py-6]   File "/home/hello-robot/ament_ws/install/stretch_web_teleop/lib/stretch_web_teleop/configure_video_streams.py", line 271, in realsense_cb
[configure_video_streams.py-6]     if self.depth_ar: img = self.pc_callback(pc, img)
[configure_video_streams.py-6]   File "/home/hello-robot/ament_ws/install/stretch_web_teleop/lib/stretch_web_teleop/configure_video_streams.py", line 155, in pc_callback
[configure_video_streams.py-6]     img[x_idx, img.shape[1] - 1 - y_idx, :] = [255, 191, 0, 50]
[configure_video_streams.py-6] IndexError: index 425 is out of bounds for axis 0 with size 424
[ERROR] [configure_video_streams.py-6]: process has died [pid 111906, exit code 1, cmd '/home/hello-robot/ament_ws/install/stretch_web_teleop/lib/stretch_web_teleop/configure_video_streams.py /home/hello-robot/ament_ws/install/stretch_web_teleop/share/stretch_web_teleop/config/configure_video_streams_params.yaml --ros-args'].

[...]

[rosbridge_websocket-11] [ERROR] [1707440191.616611396] [rosbridge_websocket]: [Client 7034eeba-86d2-4603-b3e6-d1def1da687a] [id: call_service:/depth_ar:20] call_service InvalidServiceException: Service /depth_ar does not exist
hello-amal commented 6 months ago

This will be fixed by #48 . Amongst the other changes in that PR, I filter for in-bounds pixels before coloring them.