[BUG] Stereo is not supported and no camera feed for depthai example nodes

[sonnetx]

I am using ROS humble and the default installation of depthai that comes with the turtlebot4 setup.

When I run this command (ros2 launch depthai_examples mobile_publisher.launch.py camera_model:=OAK-D-LITE) and launch rviz I get this.

[rviz2-1] [INFO] [1703337643.307606895] [rviz2]: Stereo is NOT SUPPORTED

I can't see any camera information from this node.

When I try to launch another example node with "ros2 launch depthai_examples stereo_inertial_node.launch.py"

I get the following error:

[stereo_inertial_node-2] Device USB status: SUPER [stereo_inertial_node-2] terminate called after throwing an instance of 'rclcpp::exceptions::RCLError' [stereo_inertial_node-2] what(): could not create publisher: rcl node's context is invalid, at ./src/rcl/node.c:428 [ERROR] [stereo_inertial_node-2]: process[stereo_inertial_node-2] failed to terminate '5' seconds after receiving 'SIGINT', escalating to 'SIGTERM' [INFO] [stereo_inertial_node-2]: sending signal 'SIGTERM' to process[stereo_inertial_node-2] [ERROR] [stereo_inertial_node-2]: process has died [pid 1662, exit code -6, cmd '/opt/ros/humble/lib/depthai_examples/stereo_inertial_node --ros-args --params-file /tmp/launch_params_vf9303ir --params-file /tmp/launch_params_dp82xalj --params-file /tmp/launch_params_xgmvbhrw --params-file /tmp/launch_params_w61388c1 --params-file /tmp/launch_paramssgbegky --params-file /tmp/launch_params_3466er18 --params-file /tmp/launch_params_p3txajt3 --params-file /tmp/launch_params_yx1rap94 --params-file /tmp/launch_params_7rrejbdb --params-file /tmp/launch_params_kl27do62 --params-file /tmp/launch_params_y85rz8_w --params-file /tmp/launch_params_6ur2e716 --params-file /tmp/launch_params_kmyx9yrn --params-file /tmp/launch_params_6f21byg2 --params-file /tmp/launch_params_0g_j6515 --params-file /tmp/launch_params_iu0jud_b --params-file /tmp/launch_params_4vii8ehf --params-file /tmp/launch_params_wiaqzig7 --params-file /tmp/launch_params_84bxrll2 --params-file /tmp/launch_params_pqc97tbw --params-file /tmp/launch_params_1rct54iw --params-file /tmp/launch_params_x5eicgnu --params-file /tmp/launch_params_o_jja0k7 --params-file /tmp/launch_params_r9u_wgok --params-file /tmp/launch_params_ut9ow81f --params-file /tmp/launch_params_a3d2c8y8 --params-file /tmp/launch_params_qapvngdg --params-file /tmp/launch_params75civcy --params-file /tmp/launch_params_3f8j_uyl --params-file /tmp/launch_params_3i6pw7oe --params-file /tmp/launch_params_t1m4uygs --params-file /tmp/launch_params__045z1x0 --params-file /tmp/launch_params_goautdz3 --params-file /tmp/launch_params_ev6e72cr --params-file /tmp/launch_params_galk8ysr'].

The only command that works for me is this: ros2 launch turtlebot4_bringup oakd.launch.py But even with this, stereo is not supported.

Minimal Reproducible Example Append the MRE to the bug report, instructions here

I am using the default installation of depthai that comes on the turtlebot4. Any help would be appreciated!

Which ROS version are you using? ROS 2

Which ROS distribution are you using ? humble

Please include versions of following packages - apt show ros-$ROS_DISTRO-depthai ros-$ROS_DISTRO-depthai-ros ros-$ROS_DISTRO-depthai-bridge ros-$ROS_DISTRO-depthai-ros-msgs ros-$ROS_DISTRO-depthai-ros-driver

Package: ros-humble-depthai Version: 2.23.0-1jammy.20231122.172543 Priority: optional Section: misc Maintainer: Sachin Guruswamy sachin@luxonis.com Installed-Size: 20.5 MB Depends: libc6 (>= 2.34), libgcc-s1 (>= 4.5), libopencv-core4.5d (>= 4.5.4+dfsg), libopencv-imgproc4.5d (>= 4.5.4+dfsg), libstdc++6 (>= 12), libopencv-dev, libusb-1.0-0-dev, nlohmann-json3-dev, ros-humble-ros-workspace Homepage: https://www.luxonis.com/ Download-Size: 10.6 MB APT-Sources: http://packages.ros.org/ros2-testing/ubuntu jammy/main arm64 Packages Description: DepthAI core is a C++ library which comes with firmware and an API to interact with OAK Platform

Package: ros-humble-depthai-ros Version: 2.8.2-1jammy.20231216.022314 Priority: optional Section: misc Maintainer: sachin sachin@luxonis.com Installed-Size: 43.0 kB Depends: ros-humble-depthai, ros-humble-depthai-bridge, ros-humble-depthai-descriptions, ros-humble-depthai-examples, ros-humble-depthai-filters, ros-humble-depthai-ros-driver, ros-humble-depthai-ros-msgs, ros-humble-ros-workspace Download-Size: 6,130 B APT-Manual-Installed: yes APT-Sources: http://packages.ros.org/ros2-testing/ubuntu jammy/main arm64 Packages Description: The depthai-ros package

Package: ros-humble-depthai-bridge Version: 2.8.2-1jammy.20231205.212807 Priority: optional Section: misc Maintainer: Sachin Guruswamy sachin@luxonis.com Installed-Size: 407 kB Depends: libc6 (>= 2.32), libgcc-s1 (>= 3.3.1), libopencv-calib3d4.5d (>= 4.5.4+dfsg), libopencv-core4.5d (>= 4.5.4+dfsg), libopencv-imgcodecs4.5d (>= 4.5.4+dfsg), libopencv-imgproc4.5d (>= 4.5.4+dfsg), libstdc++6 (>= 11), libboost-dev, libopencv-dev, ros-humble-camera-info-manager, ros-humble-composition-interfaces, ros-humble-cv-bridge, ros-humble-depthai, ros-humble-depthai-ros-msgs, ros-humble-image-transport, ros-humble-rclcpp, ros-humble-robot-state-publisher, ros-humble-ros-environment, ros-humble-sensor-msgs, ros-humble-std-msgs, ros-humble-stereo-msgs, ros-humble-tf2, ros-humble-tf2-geometry-msgs, ros-humble-tf2-ros, ros-humble-vision-msgs, ros-humble-xacro, ros-humble-ros-workspace Download-Size: 118 kB APT-Sources: http://packages.ros.org/ros2-testing/ubuntu jammy/main arm64 Packages Description: The depthai_bridge package

Package: ros-humble-depthai-ros-msgs Version: 2.8.2-1jammy.20231205.201618 Priority: optional Section: misc Maintainer: Sachin Guruswamy sachin@luxonis.com Installed-Size: 1,675 kB Depends: libc6 (>= 2.17), libgcc-s1 (>= 3.3.1), libpython3.10 (>= 3.10.0), libstdc++6 (>= 11), ros-humble-fastcdr, ros-humble-builtin-interfaces, ros-humble-geometry-msgs, ros-humble-rclcpp, ros-humble-rosidl-default-generators, ros-humble-sensor-msgs, ros-humble-std-msgs, ros-humble-vision-msgs, ros-humble-ros-workspace Download-Size: 127 kB APT-Sources: http://packages.ros.org/ros2-testing/ubuntu jammy/main arm64 Packages Description: Package to keep interface independent of the driver

Package: ros-humble-depthai-ros-driver Version: 2.8.2-1jammy.20231216.021048 Priority: optional Section: misc Maintainer: Adam Serafin adam.serafin@luxonis.com Installed-Size: 2,542 kB Depends: libc6 (>= 2.34), libconsole-bridge1.0 (>= 1.0.1+dfsg2), libgcc-s1 (>= 3.3.1), libopencv-core4.5d (>= 4.5.4+dfsg), libopencv-imgproc4.5d (>= 4.5.4+dfsg), libstdc++6 (>= 12), libtinyxml2-9 (>= 8.0.0), ros-humble-ament-cmake-auto, ros-humble-camera-calibration, ros-humble-cv-bridge, ros-humble-depthai, ros-humble-depthai-bridge, ros-humble-depthai-descriptions, ros-humble-depthai-examples, ros-humble-depthai-ros-msgs, ros-humble-diagnostic-msgs, ros-humble-diagnostic-updater, ros-humble-image-pipeline, ros-humble-image-transport, ros-humble-image-transport-plugins, ros-humble-pluginlib, ros-humble-rclcpp, ros-humble-rclcpp-components, ros-humble-sensor-msgs, ros-humble-std-msgs, ros-humble-std-srvs, ros-humble-vision-msgs, ros-humble-ros-workspace Download-Size: 566 kB APT-Sources: http://packages.ros.org/ros2-testing/ubuntu jammy/main arm64 Packages Description: Depthai ROS Monolithic node.

N: There are 4 additional records. Please use the '-a' switch to see them.

[Serafadam]

Hi, those warnings on rcl context seem to originate from ROS system setup, I think this might be more related to Turtlebot. Regarding oakd.launch.py, you can edit the params file here to enable rgbd mode. I don't have that much knowledge of the exact setup that is intended here so I think it would be best if you could contact the Turtlebot team.

[slowrunner]

I get those rviz2 Stereo not supported also but after doing ros2 topic list to see that what topics are being published, I added the image plugin, and set the topic to /color/image and see the rgb image, and echoing /color/mobilenet_detections the object class id(s) and confidence are displayed.

ros2 launch depthai_examples mobile_publisher.launch.py camera_model:=OAK-D-LITE

I wrote a helper to parse out just the id and conf https://github.com/slowrunner/wali_pi5/blob/main/c3ws/cmds/echo_mobilenet_detections.sh

** detections: 
class_id='9', score=0.96337890625    <<-- chair  - correct
class_id='15', score=0.91748046875   <<-- person - correct

** detections: 
class_id='9', score=0.939453125
class_id='15', score=0.74267578125

mobile_publisher worked:

   /depthai/resources/nn/mobilenet-ssd/mobilenet-ssd.json"labels":

0: "background",             
1: "aeroplane",            
2: "bicycle",            
3: "bird",            
4: "boat",            
5: "bottle",            
6: "bus",            
7: "car",            
8: "cat",           
9: "chair",            
10: "cow",           
11: "diningtable",            
12: "dog",            
13: "horse",            
14: "motorbike",            
15: "person",            
16: "pottedplant",            
17: "sheep",           
18: "sofa",            
19: "train",            
20: "tvmonitor"