luxonis / depthai-ros

Official ROS Driver for DepthAI Sensors.
MIT License
239 stars 173 forks source link

[BUG] OAK-D-LITE stereo_inertial_node.launch.py lists topics but not display color, depth nor points images #495

Closed ARLunan closed 2 days ago

ARLunan commented 4 months ago

Describe the bug OAK-D-LITE plugged into Raspberry Pi4/4G Ubuntu 22.04.1/ROS2 Humble deptha-ros ver 2.9.0 Running ros2 launch depthai_examples stereo_inertial_node.launch.py camera_model:=OAK-D-LITE. ros topic lists /color/image and /stereo/points/ , /stereo depth No color image displayed on rqt , rqt_image_view , rviz2 using stereoInertialDepthAlignROS2.rviz no PointCloud2 nor Camera image. Only urdf RobotModel of the camera

Expected behavior ros2 launch mobilenet publisher.launc.py and ros2 launch rgb_stero_node.launch.py displays;ay /color/image and /color/vieo/image & /stereo/depth on rqt_image_view

Terminal Capture

ubuntutbc@macvm-ub22h-bv:~$ ros2 launch depthai_examples stereo_inertial_node.launch.py camera_model:=OAK-D-LITE [INFO] [launch]: All log files can be found below /home/ubuntutbc/.ros/log/2024-02-08-21-08-05-202023-macvm-ub22h-bv-11478 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [robot_state_publisher-1]: process started with pid [11491] [INFO] [stereo_inertial_node-2]: process started with pid [11493] [INFO] [component_container-3]: process started with pid [11495] [INFO] [rviz2-4]: process started with pid [11497] [rviz2-4] qt.qpa.xcb: could not connect to display [rviz2-4] qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found. [rviz2-4] This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem. [rviz2-4] [rviz2-4] Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb. [rviz2-4] [robot_state_publisher-1] [INFO] [1707444485.448931238] [oak_state_publisher]: got segment oak-d-base-frame [robot_state_publisher-1] [INFO] [1707444485.449689613] [oak_state_publisher]: got segment oak-d_frame [robot_state_publisher-1] [INFO] [1707444485.450202947] [oak_state_publisher]: got segment oak_left_camera_frame [robot_state_publisher-1] [INFO] [1707444485.450568823] [oak_state_publisher]: got segment oak_left_camera_optical_frame [robot_state_publisher-1] [INFO] [1707444485.450766573] [oak_state_publisher]: got segment oak_model_origin [robot_state_publisher-1] [INFO] [1707444485.450993865] [oak_state_publisher]: got segment oak_rgb_camera_frame [robot_state_publisher-1] [INFO] [1707444485.451252323] [oak_state_publisher]: got segment oak_rgb_camera_optical_frame [robot_state_publisher-1] [INFO] [1707444485.451428948] [oak_state_publisher]: got segment oak_right_camera_frame [robot_state_publisher-1] [INFO] [1707444485.451717449] [oak_state_publisher]: got segment oak_right_camera_optical_frame [stereo_inertial_node-2] 1280 720 1280 720 [component_container-3] [INFO] [1707444485.648892763] [container]: Load Library: /opt/ros/humble/lib/libdepth_image_proc.so [component_container-3] [INFO] [1707444485.716545037] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1707444485.716619037] [container]: Instantiate class: rclcpp_components::NodeFactoryTemplate [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/convert_metric_node' in container '/container' [component_container-3] [INFO] [1707444485.736169472] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1707444485.736196556] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1707444485.736199722] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1707444485.736202431] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1707444485.736205056] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1707444485.736207722] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1707444485.736210181] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1707444485.736212681] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1707444485.736215264] [container]: Instantiate class: rclcpp_components::NodeFactoryTemplate [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/point_cloud_xyzrgb_node' in container '/container' [stereo_inertial_node-2] Listing available devices... [stereo_inertial_node-2] terminate called after throwing an instance of 'std::runtime_error' [stereo_inertial_node-2] what(): " DepthAI Device with MxId "x" not found. " [ERROR] [rviz2-4]: process has died [pid 11497, exit code -6, cmd '/opt/ros/humble/lib/rviz2/rviz2 --display-config /opt/ros/humble/share/depthai_examples/rviz/stereoInertialDepthAlignROS2.rviz --ros-args']. [ERROR] [stereo_inertial_node-2]: process has died [pid 11493, exit code -6, cmd '/opt/ros/humble/lib/depthai_examples/stereo_inertial_node --ros-args --params-file /tmp/launch_params_sjdz0415 --params-file /tmp/launch_params_c17wlj3f --params-file /tmp/launch_params_2scf_5uj --params-file /tmp/launch_params_f5h8hxfh --params-file /tmp/launch_params_5mq7bc5o --params-file /tmp/launch_params_oaa3ildz --params-file /tmp/launch_params_6ioofrt5 --params-file /tmp/launch_params_1swfl_j7 --params-file /tmp/launch_params_pgz77mdq --params-file /tmp/launch_params_xi4a5e6e --params-file /tmp/launch_params_p7g1pvlx --params-file /tmp/launch_params_1agm52r5 --params-file /tmp/launch_params_lb1ihwrn --params-file /tmp/launch_params_slwyzvhp --params-file /tmp/launch_params_zabd8pj3 --params-file /tmp/launch_params_ftp4p43z --params-file /tmp/launch_params_pg0h8_9j --params-file /tmp/launch_params_oakaqo4k --params-file /tmp/launch_params_v9hda9gt --params-file /tmp/launch_params_mbrdtvk2 --params-file /tmp/launch_params_e0jk3tkm --params-file /tmp/launch_params_vaeo2q7e --params-file /tmp/launch_params_0rl564io --params-file /tmp/launch_params_odiiw84y --params-file /tmp/launch_params_dqqdiylp --params-file /tmp/launch_params_uv6zuxhp --params-file /tmp/launch_params_mtdcz9e8 --params-file /tmp/launch_params_cv_puy13 --params-file /tmp/launch_params_ps2tfc5j --params-file /tmp/launch_params_02vae12p --params-file /tmp/launch_params_7qfm587v --params-file /tmp/launch_params_1q246hqn --params-file /tmp/launch_params_isdbanj8 --params-file /tmp/launch_params_saojzklh --params-file /tmp/launch_params_e9bziyi3']. [component_container-3] [WARN] [1707444900.277069490] [point_cloud_xyzrgb_node]: New subscription discovered on topic '/stereo/points', requesting incompatible QoS. No messages will be sent to it. Last incompatible policy: RELIABILITY_QOS_POLICY [component_container-3] [WARN] [1707444900.278032619] [point_cloud_xyzrgb_node]: New subscription discovered on topic '/stereo/points', requesting incompatible QoS. No messages will be sent to it. Last incompatible policy: RELIABILITY_QOS_POLICY ^C[WARNING] [launch]: user interrupted with ctrl-c (SIGINT)

[INFO] [robot_state_publisher-1]: process has finished cleanly [pid 11491] [INFO] [component_container-3]: process has finished cleanly [pid 11495]

ubuntutbc@macvm-ub22h-bv:~$

Attach system log

Additional context ros2 launch depthai_examples stereo.launch.py doesn't display /color/image either

Serafadam commented 4 months ago

Hi, sorry for the delay and thanks for the report. It seems that there has been a small change in firmware side regarding IMU detection. While stereo_inertial_node and other nodes in depthai_examples serve only as showcases how to write custom nodes, depthai_ros_driver has been updated, for now the fix is available on this branch, and will be merged to the upstream soon.

ARLunan commented 4 months ago

Thank you for the response. A ros2 build for source question: To continue using the depthai-ros examples in my experimenting until the upstream binary install is released , can I configure a workspace with the src depthai-ros and designate only the depthai-ros-drivers for a build compile as an overlay. Or is it necessary to build the entire depthai-ros repository package as an overlay?

ARLunan commented 4 months ago

On Raspberry Pi 4/4G /Ubuntu 22.04.3 ROS 2 Humble, After clone & rosdep-All required rosdeps installed successfully: Error message on build the new branch https://github.com/luxonis/depthai-ros/tree/no_imu_fix , rosdepthai_ros_driver package.

ubuntu@rp4-ub22h-mt:~/depthai-ros_ws$ colcon build --symlink-install --packages-select depthai_ros_driver --executor sequential --parallel-workers 1 [5.704s] WARNING:colcon.colcon_core.package_selection:Some selected packages are already built in one or more underlay workspaces: 'depthai_ros_driver' is in: /opt/ros/humble If a package in a merged underlay workspace is overridden and it installs headers, then all packages in the overlay must sort their include directories by workspace order. Failure to do so may result in build failures or undefined behavior at run time. If the overridden package is used by another package in any underlay, then the overriding package in the overlay must be API and ABI compatible or undefined behavior at run time may occur.

If you understand the risks and want to override a package anyways, add the following to the command line: --allow-overriding depthai_ros_driver

This may be promoted to an error in a future release of colcon-override-check. Starting >>> depthai_ros_driver [6.151s] WARNING:colcon.colcon_core.shell:The following packages are in the workspace but haven't been built:

Summary: 0 packages finished [39.6s] 1 package failed: depthai_ros_driver 1 package had stderr output: depthai_ros_driver ubuntu@rp4-ub22h-mt:~/depthai-ros_ws$ cd src

Serafadam commented 4 months ago

Hi, this error message indicates that you should build depthai_bridge package as well since it also had a small modification in the latest release and depthai_ros_driver depends on it. Depending on changes between different releases some additional packages require to be also built. With each release we also provide Docker containers with pre-built workspace that can be used for testing/development purposes, more information here

ARLunan commented 4 months ago

Thanks for the response, which enabled a successful colcon build, and I now have your revised packages installed. Unfortunately the results are the same with no image or rviz pointcloud displays on several launch scripts even though the image , depth and points topics are published.So I prefer to wait until your upstream Binary install packages are available. Though If I get the opportunity, I may try the Docker ( which I am a new, inexperienced user.) as a learning experience,

ARLunan commented 4 months ago

I note Issue #500 and no_imu_fix branch is removed, which I assume fixes this issue #485 in depthai_ros_drivers? To try this I need to update this packge with a build? Should I build any others as well?

ARLunan commented 4 months ago

Ok, so I performed the 2.9.0-1 update of the this new Feb 23/24 binary install. All 4 launch files run. _mobilepublisher.launch.py continues to work displaying color image on rqt_image_view. _rgb_stereonode.launch.py displays RGB and Depth on rqt_image_view. stereo.launch.py enables the display of the BW mono and color images on rqt_image_view and BW image & pointcloud2 on rviz2. One small error for the Pointcloud2 Channel Name = Intensity for display a colored points. stereo.launch.py opens rviz displays BW image_rect and pointcloud2 and BW image_rect & depth in rqt_image_view. All good. The problem: rgb_stereo_inertialnode.launch where rviz2 launches, and topics listed /color/image, /stereo/depth, /stereo/points but no rgb, depth images show in rqt_image_view, rqt & no color, depth nor points display in rviz2. I tried all 3 rviz files with no different results. Here is the last parts of the launch log that states there is no messages sent to /stereo/points; [component_container-3] [WARN] [1708727943.675178994] [point_cloud_xyzrgb_node]: New subscription discovered on topic '/stereo/points', requesting incompatible QoS. No messages will be sent to it. Last incompatible policy: RELIABILITY_QOS_POLICY [component_container-3] [WARN] [1708727943.680064118] [point_cloud_xyzrgb_node]: New subscription discovered on topic '/stereo/points', requesting incompatible QoS. No messages will be sent to it. Last incompatible policy: RELIABILITY_QOS_POLICY

Serafadam commented 4 months ago

Hi, could you check if there is data published on those topics via ros2 topic hz/echo topic_name?

ARLunan commented 4 months ago

Using Synaptic confirmed installed depthai-ros version is 2.9.0-1 jammy.2024

1) mobile_publisher.launch.py Shows image 2) stereo_inertail_nodelaunch.pt shows IMU error and no image or echo topic

1) ros2 launch depthai_examples mobile_publisher.launch.py camera_model:=OAK-D-LITE rqt_image_view shows Color Image ---Good! ubuntu@rp4-ub22h-r2:~$ ros2 topic list|grep color /color/camera_info /color/image /color/image/compressed /color/image/compressedDepth /color/image/theora /color/mobilenet_detections ubuntu@rp4-ub22h-r2:~$ ros2 topic echo /color/image header: stamp: sec: 1709150838 nanosec: 844661940 frame_id: oak_rgb_camera_optical_frame height: 300 width: 300 encoding: bgr8 is_bigendian: 0 step: 900 data:

2) Running stereo_inertial_node launch throws a "IMU not detected" error ubuntu@rp4-ub22h-r2:~$ ros2 launch depthai_examples stereo_inertial_node.launch.py camera_model:=OAK-D-LITE enableRviz:=False [INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2024-02-28-15-27-37-378150-rp4-ub22h-r2-6872 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [robot_state_publisher-1]: process started with pid [6885] [INFO] [stereo_inertial_node-2]: process started with pid [6887] [INFO] [component_container-3]: process started with pid [6889] [robot_state_publisher-1] [INFO] [1709152058.685191088] [oak_state_publisher]: got segment oak-d-base-frame [robot_state_publisher-1] [INFO] [1709152058.685571216] [oak_state_publisher]: got segment oak-d_frame [robot_state_publisher-1] [INFO] [1709152058.685659568] [oak_state_publisher]: got segment oak_left_camera_frame [robot_state_publisher-1] [INFO] [1709152058.685703530] [oak_state_publisher]: got segment oak_left_camera_optical_frame [robot_state_publisher-1] [INFO] [1709152058.685738326] [oak_state_publisher]: got segment oak_model_origin [robot_state_publisher-1] [INFO] [1709152058.685767974] [oak_state_publisher]: got segment oak_rgb_camera_frame [robot_state_publisher-1] [INFO] [1709152058.685798511] [oak_state_publisher]: got segment oak_rgb_camera_optical_frame [robot_state_publisher-1] [INFO] [1709152058.685828271] [oak_state_publisher]: got segment oak_right_camera_frame [robot_state_publisher-1] [INFO] [1709152058.685857696] [oak_state_publisher]: got segment oak_right_camera_optical_frame [component_container-3] [INFO] [1709152058.800815871] [container]: Load Library: /opt/ros/humble/lib/libdepth_image_proc.so [component_container-3] [INFO] [1709152059.181596248] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1709152059.183331333] [container]: Instantiate class: rclcpp_components::NodeFactoryTemplate [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/convert_metric_node' in container '/container' [component_container-3] [INFO] [1709152059.282435173] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1709152059.282572524] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1709152059.282607506] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1709152059.282635191] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1709152059.282661320] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1709152059.282686820] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1709152059.282712783] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1709152059.282737579] [container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-3] [INFO] [1709152059.282762746] [container]: Instantiate class: rclcpp_components::NodeFactoryTemplate [stereo_inertial_node-2] 1280 720 1280 720 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/point_cloud_xyzrgb_node' in container '/container' [stereo_inertial_node-2] Listing available devices... [stereo_inertial_node-2] Device Mx ID: 1844301061369F1200 [stereo_inertial_node-2] [1844301061369F1200] [1.1.2] [1.062] [MonoCamera(2)] [error] OV7251 only supports THE_480_P/THE_400_P resolutions, defaulting to THE_480_P [stereo_inertial_node-2] [1844301061369F1200] [1.1.2] [1.062] [MonoCamera(1)] [error] OV7251 only supports THE_480_P/THE_400_P resolutions, defaulting to THE_480_P [stereo_inertial_node-2] terminate called after throwing an instance of 'std::runtime_error' [stereo_inertial_node-2] what(): IMU(5) - IMU invalid settings!: IMU not detected. Your board doesn't have IMU or requires firmware update (Set imu.enableFirmwareUpdate(True) explicitly.). 1, 321 [ERROR] [stereo_inertial_node-2]: process has died [pid 6887, exit code -6, cmd '/opt/ros/humble/lib/depthai_examples/s

Serafadam commented 3 months ago

Hi, thanks for the information, could you also post results from running:

ARLunan commented 3 months ago

ros2 launch depthai_ros_driver camera.launch.py ubuntu@rp4-ub22h-r2:~$ ros2 launch depthai_ros_driver camera.launch.py [INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2024-03-04-08-45-25-974333-rp4-ub22h-r2-4139 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [component_container-1]: process started with pid [4153] [component_container-1] [INFO] [1709559927.571233108] [oak_container]: Load Library: /opt/ros/humble/lib/librobot_state_publisher_node.so [component_container-1] [INFO] [1709559927.643403055] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709559927.643668274] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709559927.715397264] [oak_state_publisher]: got segment oak [component_container-1] [INFO] [1709559927.715603428] [oak_state_publisher]: got segment oak-d-base-frame [component_container-1] [INFO] [1709559927.715661668] [oak_state_publisher]: got segment oak_imu_frame [component_container-1] [INFO] [1709559927.715718889] [oak_state_publisher]: got segment oak_left_camera_frame [component_container-1] [INFO] [1709559927.715769851] [oak_state_publisher]: got segment oak_left_camera_optical_frame [component_container-1] [INFO] [1709559927.715814962] [oak_state_publisher]: got segment oak_model_origin [component_container-1] [INFO] [1709559927.715857961] [oak_state_publisher]: got segment oak_rgb_camera_frame [component_container-1] [INFO] [1709559927.715902239] [oak_state_publisher]: got segment oak_rgb_camera_optical_frame [component_container-1] [INFO] [1709559927.715946608] [oak_state_publisher]: got segment oak_right_camera_frame [component_container-1] [INFO] [1709559927.715992460] [oak_state_publisher]: got segment oak_right_camera_optical_frame [component_container-1] [INFO] [1709559927.737840743] [oak_container]: Load Library: /opt/ros/humble/lib/libdepthai_ros_driver.so [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/oak_state_publisher' in container 'oak_container' [component_container-1] [INFO] [1709559929.112805564] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709559929.113018061] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709559929.178945598] [oak]: No ip/mxid specified, connecting to the next available device. [component_container-1] [INFO] [1709559932.021931556] [oak]: Camera with MXID: 1844301061369F1200 and Name: 1.1.2 connected! [component_container-1] [INFO] [1709559932.023686570] [oak]: USB SPEED: SUPER [component_container-1] [INFO] [1709559932.051532780] [oak]: Device type: OAK-D-LITE [component_container-1] [INFO] [1709559932.054704850] [oak]: Pipeline type: RGBD [component_container-1] [INFO] [1709559932.179519109] [oak]: NN Family: mobilenet [component_container-1] [INFO] [1709559932.547953576] [oak]: NN input size: 300 x 300. Resizing input image in case of different dimensions. [component_container-1] [INFO] [1709559933.264023312] [oak]: Finished setting up pipeline. [component_container-1] [ERROR] [1709559935.016917243] [oak_container]: Component constructor threw an exception: IMU(12) - IMU invalid settings!: IMU not detected. Your board doesn't have IMU or requires firmware update (Set imu.enableFirmwareUpdate(True) explicitly.). 0, 328 [ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'oak' of type 'depthai_ros_driver::Camera' in container '/oak_container': Component constructor threw an exception: IMU(12) - IMU invalid settings!: IMU not detected. Your board doesn't have IMU or requires firmware update (Set imu.enableFirmwareUpdate(True) explicitly.). 0, 328

ros2 launch depthai_ros_driver rgbd_pcl.launch.py ubuntu@rp4-ub22h-r2:~$ ros2 launch depthai_ros_driver rgbd_pcl.launch.py [INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2024-03-04-08-47-14-482235-rp4-ub22h-r2-4257 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [component_container-1]: process started with pid [4272] [component_container-1] [INFO] [1709560035.698564429] [oak_container]: Load Library: /opt/ros/humble/lib/librobot_state_publisher_node.so [component_container-1] [INFO] [1709560035.711716015] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560035.711867384] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560035.737569859] [oak_state_publisher]: got segment oak [component_container-1] [INFO] [1709560035.737712432] [oak_state_publisher]: got segment oak-d-base-frame [component_container-1] [INFO] [1709560035.737745395] [oak_state_publisher]: got segment oak_imu_frame [component_container-1] [INFO] [1709560035.737774524] [oak_state_publisher]: got segment oak_left_camera_frame [component_container-1] [INFO] [1709560035.737802894] [oak_state_publisher]: got segment oak_left_camera_optical_frame [component_container-1] [INFO] [1709560035.737828690] [oak_state_publisher]: got segment oak_model_origin [component_container-1] [INFO] [1709560035.737853431] [oak_state_publisher]: got segment oak_rgb_camera_frame [component_container-1] [INFO] [1709560035.737878282] [oak_state_publisher]: got segment oak_rgb_camera_optical_frame [component_container-1] [INFO] [1709560035.737903338] [oak_state_publisher]: got segment oak_right_camera_frame [component_container-1] [INFO] [1709560035.737927967] [oak_state_publisher]: got segment oak_right_camera_optical_frame [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/oak_state_publisher' in container 'oak_container' [component_container-1] [INFO] [1709560035.745283216] [oak_container]: Load Library: /opt/ros/humble/lib/libdepthai_ros_driver.so [component_container-1] [INFO] [1709560036.027595426] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560036.027771962] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560036.073894516] [oak]: No ip/mxid specified, connecting to the next available device. [component_container-1] [INFO] [1709560038.882587974] [oak]: Camera with MXID: 1844301061369F1200 and Name: 1.1.2 connected! [component_container-1] [INFO] [1709560038.886596586] [oak]: USB SPEED: SUPER [component_container-1] [INFO] [1709560038.917678868] [oak]: Device type: OAK-D-LITE [component_container-1] [INFO] [1709560038.921260152] [oak]: Pipeline type: RGBD [component_container-1] [INFO] [1709560040.082296171] [oak]: Finished setting up pipeline. [component_container-1] [ERROR] [1709560041.882807391] [oak_container]: Component constructor threw an exception: IMU(9) - IMU invalid settings!: IMU not detected. Your board doesn't have IMU or requires firmware update (Set imu.enableFirmwareUpdate(True) explicitly.). 0, 328 [ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'oak' of type 'depthai_ros_driver::Camera' in container '/oak_container': Component constructor threw an exception: IMU(9) - IMU invalid settings!: IMU not detected. Your board doesn't have IMU or requires firmware update (Set imu.enableFirmwareUpdate(True) explicitly.). 0, 328 [component_container-1] [INFO] [1709560041.885179000] [oak_container]: Load Library: /opt/ros/humble/lib/libdepth_image_proc.so [component_container-1] [INFO] [1709560041.918895188] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560041.919064168] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560041.919102353] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560041.919131112] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560041.919158500] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560041.919184370] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560041.919208870] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560041.919233685] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560041.919258555] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/point_cloud_xyzrgb_node' in container 'oak_container'

ros2 launch depthai_filters spatial_bb.launch.p buntu@rp4-ub22h-r2:~$ ros2 launch depthai_filters spatial_bb.launch.py [INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2024-03-04-08-48-21-687875-rp4-ub22h-r2-4366 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [component_container-1]: process started with pid [4384] [component_container-1] [INFO] [1709560102.906629451] [oak_container]: Load Library: /opt/ros/humble/lib/librobot_state_publisher_node.so [component_container-1] [INFO] [1709560102.921478885] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560102.921622828] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560102.948941304] [oak_state_publisher]: got segment oak [component_container-1] [INFO] [1709560102.949081748] [oak_state_publisher]: got segment oak-d-base-frame [component_container-1] [INFO] [1709560102.949117081] [oak_state_publisher]: got segment oak_imu_frame [component_container-1] [INFO] [1709560102.949144618] [oak_state_publisher]: got segment oak_left_camera_frame [component_container-1] [INFO] [1709560102.949170599] [oak_state_publisher]: got segment oak_left_camera_optical_frame [component_container-1] [INFO] [1709560102.949196728] [oak_state_publisher]: got segment oak_model_origin [component_container-1] [INFO] [1709560102.949222747] [oak_state_publisher]: got segment oak_rgb_camera_frame [component_container-1] [INFO] [1709560102.949248173] [oak_state_publisher]: got segment oak_rgb_camera_optical_frame [component_container-1] [INFO] [1709560102.949273135] [oak_state_publisher]: got segment oak_right_camera_frame [component_container-1] [INFO] [1709560102.949297135] [oak_state_publisher]: got segment oak_right_camera_optical_frame [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/oak_state_publisher' in container 'oak_container' [component_container-1] [INFO] [1709560102.958065333] [oak_container]: Load Library: /opt/ros/humble/lib/libdepthai_ros_driver.so [component_container-1] [INFO] [1709560103.241254125] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560103.241463549] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560103.287971115] [oak]: No ip/mxid specified, connecting to the next available device. [component_container-1] [INFO] [1709560106.090233402] [oak]: Camera with MXID: 1844301061369F1200 and Name: 1.1.2 connected! [component_container-1] [INFO] [1709560106.094337615] [oak]: USB SPEED: SUPER [component_container-1] [INFO] [1709560106.122954511] [oak]: Device type: OAK-D-LITE [component_container-1] [INFO] [1709560106.128684490] [oak]: Pipeline type: RGBD [component_container-1] [INFO] [1709560106.242920835] [oak]: NN Family: mobilenet [component_container-1] [INFO] [1709560107.493982827] [oak]: Finished setting up pipeline. [component_container-1] [ERROR] [1709560109.086678175] [oak_container]: Component constructor threw an exception: IMU(14) - IMU invalid settings!: IMU not detected. Your board doesn't have IMU or requires firmware update (Set imu.enableFirmwareUpdate(True) explicitly.). 0, 328 [ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'oak' of type 'depthai_ros_driver::Camera' in container '/oak_container': Component constructor threw an exception: IMU(14) - IMU invalid settings!: IMU not detected. Your board doesn't have IMU or requires firmware update (Set imu.enableFirmwareUpdate(True) explicitly.). 0, 328 [component_container-1] [INFO] [1709560109.089254750] [oak_container]: Load Library: /opt/ros/humble/lib/librectify.so [component_container-1] [INFO] [1709560109.099315552] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560109.099462070] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560109.204822150] [oak_container]: Load Library: /opt/ros/humble/lib/libdepth_image_proc.so [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/rectify_color_node' in container 'oak_container' [component_container-1] [INFO] [1709560109.214014532] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560109.214143976] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560109.214176827] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560109.214203049] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560109.214228568] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560109.214253845] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560109.214278845] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560109.214303123] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560109.214327530] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/point_cloud_xyzrgb_node' in container 'oak_container' [component_container-1] [INFO] [1709560109.238562013] [oak_container]: Load Library: /opt/ros/humble/lib/libdepthai_filters.so [component_container-1] [INFO] [1709560109.367039288] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560109.367180602] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560109.367211732] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560109.367238713] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560109.367264768] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1709560109.367290787] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/spatial_bb_node' in container 'oak_container'

ARLunan commented 3 months ago

Per your request, posted terminal results for the 3 launch files. Evidence of IMU finding errors.May I ask another somewhat related question related to the cam-x parametersUsing the depthai-examples with an Oak-D-Lite, what is the orientation  of the launch file xyzrpy parameters relative to the case physical orientation. The ROS convention for robot models is FLU x=fwd y=left z=up. What direction is the camera xyz relative to its case.? Is it RDF x=down y=down z=fwd?Sent from Ross LunanOn Mar 4, 2024, at 3:12 AM, Adam Serafin @.***> wrote: Hi, thanks for the information, could you also post results from running:

ros2 launch depthai_ros_driver camera.launch.py ros2 launch depthai_ros_driver rgbd_pcl.launch.py ros2 launch depthai_filters spatial_bb.launch.py

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Serafadam commented 3 months ago

Hi, the fixes for IMU are on the main branch and should be released with the start of the new week. Regarding frame orientation, I am not sure if I exactly understand, roll, pitch and yaw in the launch file are used to orient the camera for attaching it to an external frame. Regarding the sensors themselves, you have a base sensor frame which uses ROS convention and an "optical" frame which corresponds to OpenCV one.

ARLunan commented 3 months ago

Adam, Thank you for this, look forward to trying the Release and glad to be of assistance in tracking down bugs. I'm a very minor Oak-D-Lite user, but am a participate in several Robot forums where this Camera is discussed. I hope to use this particular launch and ROS Driver file in my Robot vehicle designs for generating and using the point data for object tracking. Ross


From: Adam Serafin @.> Sent: Thursday, March 7, 2024 9:28 AM To: luxonis/depthai-ros @.> Cc: RossBots @.>; Author @.> Subject: Re: [luxonis/depthai-ros] [BUG] OAK-D-LITE stereo_inertial_node.launch.py lists topics but not display color, depth nor points images (Issue #495)

Hi, the fixes for IMU are on the main branch and should be released with the start of the new week. Regarding frame orientation, I am not sure if I exactly understand, roll, pitch and yaw in the launch file are used to orient the camera for attaching it to an external frame. Regarding the sensors themselves, you have a base sensor frame which uses ROS convention and an "optical" frame which corresponds to OpenCV one.

— Reply to this email directly, view it on GitHubhttps://github.com/luxonis/depthai-ros/issues/495#issuecomment-1983621238, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAGPD4BEK6ERWGDHTMFE6R3YXB2PLAVCNFSM6AAAAABDAY57S2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBTGYZDCMRTHA. You are receiving this because you authored the thread.Message ID: @.***>

ARLunan commented 1 week ago

After a complete ros humble apt upgrade that included depthai-ros upgrade today, June 22, 2024 which installed several upgrades , still getting the IMU error running the stereo-inertial launch file. Other examples work ok. I tried a calibration factory reset displaying version 0 00 but unable reset. Appreciate troubleshooting suggestions.

ARLunan commented 5 days ago

Seems, my imu detect problem is commented here related to perhaps older OakD cameras, such as mine which was a kickstarter version. (https://discuss.luxonis.com/d/3422-depthai-ros-pointcloud-with-no-imu-oak-d-lite) . Can u suggest a means to set a param to force ignoring imu checking in the launch file?

ARLunan commented 4 days ago

Unresolved with Release 2.9.0-1 launching starts up and throws a error. ros2 launch depthai_examples stereo_inertial_node.launch.py camera_model:=OAK-D-LITE enableRviz:=False [INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2024-06-26-19-21-09-439160-rp4-ub22h-mt-5755 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [robot_state_publisher-1]: process started with pid [5768] [INFO] [stereo_inertial_node-2]: process started with pid [5770] [INFO] [component_container-3]: process started with pid [5772] [robot_state_publisher-1] [INFO] [1719444070.769598717] [oak_state_publisher]: got segment oak-d-base-frame . . [stereo_inertial_node-2] what(): IMU(5) - IMU invalid settings!: IMU not detected. Your board doesn't have IMU or requires firmware update (Set imu.enableFirmwareUpdate(True) explicitly.). 1, 347 [ERROR] [stereo_inertial_node-2]: process has died [pid 5770, exit code -6, cmd '/opt/ros/humble/lib/depthai_examples/stereo_inertial_node --ros-args --params-file /tmp

Remaining nodes: ubuntutbc@macvm-ub22h-bt:~$ ros2 node list /container /convert_metric_node /launch_ros_5755 /oak_state_publisher /point_cloud_xyzrgb_node

No camera image displayed on rqt_image_view So closing this #495 and posting Issue on Forum

Serafadam commented 3 days ago

Hi, sorry for the delay, could you try running camera node (ros2 launch depthai_ros_driver camera.launch.py) while setting parameter camera.i_enable_imu: false?

ARLunan commented 3 days ago

Oak-D Lite ros2 launch depthai_ros_driver camera.launch.py camera.i_enable_imu:=false

ros2 launch depthai_ros_driver camera.launch.py camera.I_enable_imu:=false [INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2024-06-27-13-18-24-198729-rp4-ub22h-mt-10127 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [component_container-1]: process started with pid [10141] [component_container-1] [INFO] [1719508705.739391560] [oak_container]: Load Library: /opt/ros/humble/lib/librobot_state_publisher_node.so [component_container-1] [INFO] [1719508705.754612024] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1719508705.754759246] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate [component_container-1] [INFO] [1719508705.785157045] [oak_state_publisher]: got segment oak . . [component_container-1] [INFO] [1719509489.036545049] [oak]: Finished setting up pipeline. [component_container-1] [ERROR] [1719509491.867887049] [oak_container]: Component constructor threw an exception: IMU(12) - IMU invalid settings!: IMU not detected. Your board doesn't have IMU or requires firmware update (Set imu.enableFirmwareUpdate(True) explicitly.). 0, 354 [ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'oak' of type 'depthai_ros_driver::Camera' in container '/oak_container': Component constructor threw an exception: IMU(12) - IMU invalid settings!: IMU not detected. Your board doesn't have IMU or requires firmware update (Set imu.enableFirmwareUpdate(True) explicitly.). 0, 354

*** NOTE NO Images Published

ubuntu@rp4-ub22h-mt:~$ ros2 node list /launch_ros_9849 /oak_container /oak_state_publisher ubuntu@rp4-ub22h-mt:~$ ros2 topic list /joint_states /parameter_events /robot_description /rosout /tf /tf_static ubuntu@rp4-ub22h-mt:~$ ros2 param dump /oak_state_publisher /oak_state_publisher: ros__parameters: frame_prefix: '' ignore_timestamp: false publish_frequency: 20.0 qos_overrides: /joint_states: subscription: depth: 5 history: keep_last reliability: best_effort /parameter_events: publisher: depth: 1000 durability: volatile history: keep_last reliability: reliable /tf: publisher: depth: 100 durability: volatile history: keep_last reliability: reliable /tf_static: publisher: depth: 1 history: keep_last reliability: reliable robot_description: <?xml version="1.0" ?> <origin rpy="0.0 0.0 0.0" xyz="0.0 0.0 0.0"/>

use_sim_time: false ubuntu@rp4-ub22h-mt:~$ ros2 param dump /oak_container ^Cubuntu@rp4-ub22h-mt:~$ ros2 param dump /launch_ros_9849 /launch_ros_9849: ros__parameters: use_sim_time: false ubuntu@rp4-ub22h-mt:~$
ARLunan commented 3 days ago

Some Progress with using a specific params_file:=no_imu on my Desktop /oak: ros__parameters: camera: i_enable_imu: false i_enable_ir: false ros2 launch depthai_ros_driver camera.launch.py model:=OAK-D-LITE params_file:=/home/ubuntu/Desktop/no_imu.yaml [INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2024-06-27-15-25-32-137462-rp4-ub22h-mt-6754 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [component_container-1]: process started with pid [6768] [component_container-1] [INFO] [1719516333.590873623] [oak_container]: Load Library: /opt/ros/humble/lib/librobot_state_publisher_node.so . . . component_container-1] [INFO] [1719516337.189922001] [oak]: Device type: OAK-D-LITE [component_container-1] [INFO] [1719516337.193936588] [oak]: Pipeline type: RGBD [component_container-1] [INFO] [1719516337.309804026] [oak]: NN Family: mobilenet [component_container-1] [INFO] [1719516337.515792725] [oak]: NN input size: 300 x 300. Resizing input image in case of different dimensions. [component_container-1] [INFO] [1719516337.668190713] [oak]: Finished setting up pipeline. [component_container-1] [INFO] [1719516340.787822902] [oak]: Camera ready! [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/oak' in container '/oak_container'

Nodes & Topics ubuntu@rp4-ub22h-mt:~$ ros2 node list /launch_ros_4939 /oak /oak_container /oak_state_publisher ubuntu@rp4-ub22h-mt:~$ ros2 topic list /diagnostics /joint_states /oak/nn/spatial_detections /oak/rgb/camera_info /oak/rgb/image_raw /oak/rgb/image_raw/compressed /oak/rgb/image_raw/compressedDepth /oak/rgb/image_raw/theora /oak/stereo/camera_info /oak/stereo/image_raw /oak/stereo/image_raw/compressed /oak/stereo/image_raw/compressedDepth /oak/stereo/image_raw/theora /parameter_events /robot_description /rosout /tf /tf_static

$ rqt displays the images -- Yeh !


Bad news still a stereo_inertial_node running the following script does not seem to pick up the no_imu.yaml

ros2 launch depthai_examples stereo_inertial_node.launch.py camera_model:=OAK-D-LITE enableRviz:=False params_file:=/home/ubuntu/Desktop/no_imu.yaml [INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2024-06-27-15-21-48-750362-rp4-ub22h-mt-6625 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [robot_state_publisher-1]: process started with pid [6638] [INFO] [stereo_inertial_node-2]: process started with pid [6640] [INFO] [component_container-3]: process started with pid [6642] [robot_state_publisher-1] [INFO] [1719516110.040157263] [oak_state_publisher]: got segment oak-d-base-frame [robot_state_publisher-1] [INFO] [1719516110.040532813] [oak_state_publisher]: got segment oak-d_frame . . . [stereo_inertial_node-2] what(): IMU(5) - IMU invalid settings!: IMU not detected. Your board doesn't have IMU or requires firmware update (Set imu.enableFirmwareUpdate(True) explicitly.). 1, 347 [ERROR] [stereo_inertial_node-2]: process has died [pid 6640, exit code -6, cmd '/opt/ros/humble/lib/depthai_examples/stereo_inertial_node --ros-args --params-file /tmp/launch_params_68bgszd4 --params-file /tmp/launch_params_pxlewkof --params-fi blah blah blah

^C[WARNING] [launch]: user interrupted with ctrl-c (SIGINT)

[INFO] [robot_state_publisher-1]: process has finished cleanly [pid 6638] [INFO] [component_container-3]: process has finished cleanly [pid 6642]

ubuntu@rp4-ub22h-mt:~/Desktop$

ARLunan commented 3 days ago

Another Question regarding the [Depthai] ROS Driver Docs: [https://docs.luxonis.com/software/ros/depthai-ros/driver/] What is the script that displays the "Setting RGB Parameters" ?

ARLunan commented 2 days ago

I believe I can close this issue. A recent post on the Luxonis ROS forum stated that "Can't use stereo inertial node without IMU since that IMU is literally in the node name." I missed that obvious characterization. I also checked the stereo_inertial_publisher.cpp code where it appears that the code expects an IMU in the camera and there is no smooth check otherwise other than crashing. I can use one of the other excellent Camera packages that publish depth topics for my robot application. Appreciate your assistance with getting the camera.launch.py to work using an no_imu.yaml.