No aligned depth topic Humble/OAK-D PRO W POE

[Victorsoeby]

Hi.

It seems that the depth align does not work for OAK-D-PRO-W POE and the Humble branch, as i am getting this from the camera.launch.py file:

root@victor-Legion-5-Pro-16ACH6H:/# ros2 launch depthai_ros_driver camera.launch.py 
[INFO] [launch]: All log files can be found below /root/.ros/log/2023-04-28-09-57-33-757235-victor-Legion-5-Pro-16ACH6H-178
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [robot_state_publisher-1]: process started with pid [192]
[INFO] [component_container-2]: process started with pid [194]
[robot_state_publisher-1] [INFO] [1682675853.922572649] [oak_state_publisher]: got segment oak
[robot_state_publisher-1] [INFO] [1682675853.922658059] [oak_state_publisher]: got segment oak-d-base-frame
[robot_state_publisher-1] [INFO] [1682675853.922682363] [oak_state_publisher]: got segment oak_imu_frame
[robot_state_publisher-1] [INFO] [1682675853.922702685] [oak_state_publisher]: got segment oak_left_camera_frame
[robot_state_publisher-1] [INFO] [1682675853.922721262] [oak_state_publisher]: got segment oak_left_camera_optical_frame
[robot_state_publisher-1] [INFO] [1682675853.922740048] [oak_state_publisher]: got segment oak_model_origin
[robot_state_publisher-1] [INFO] [1682675853.922757786] [oak_state_publisher]: got segment oak_rgb_camera_frame
[robot_state_publisher-1] [INFO] [1682675853.922775804] [oak_state_publisher]: got segment oak_rgb_camera_optical_frame
[robot_state_publisher-1] [INFO] [1682675853.922793054] [oak_state_publisher]: got segment oak_right_camera_frame
[robot_state_publisher-1] [INFO] [1682675853.922810164] [oak_state_publisher]: got segment oak_right_camera_optical_frame
[component_container-2] [INFO] [1682675854.153322066] [oak_container]: Load Library: /ws/install/lib/libdepthai_ros_driver.so
[component_container-2] [INFO] [1682675854.189981076] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate<depthai_ros_driver::Camera>
[component_container-2] [INFO] [1682675854.190084504] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<depthai_ros_driver::Camera>
[component_container-2] [INFO] [1682675854.195066242] [oak]: No ip/mxid specified, connecting to the next available device.
[component_container-2] [INFO] [1682675860.937841715] [oak]: Camera with MXID: 184430109159A30F00 and Name: 10.3.3.66 connected!
[component_container-2] [INFO] [1682675860.937984881] [oak]: PoE camera detected. Consider enabling low bandwidth for specific image topics (see readme).
[component_container-2] [INFO] [1682675860.960688140] [oak]: Device type: OAK-D-PRO-W-POE-97
[component_container-2] [INFO] [1682675860.965839795] [oak]: Pipeline type: RGBD
[component_container-2] [INFO] [1682675860.981345111] [oak]: NN Family: mobilenet
[component_container-2] [INFO] [1682675861.029812792] [oak]: NN input size: 300 x 300. Resizing input image in case of different dimensions.
[component_container-2] [INFO] [1682675861.068007685] [oak]: Finished setting up pipeline.
[component_container-2] [INFO] [1682675862.255802479] [oak]: Camera ready!
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/oak' in container '/oak_container'
[component_container-2] [184430109159A30F00] [10.3.3.66] [7.224] [StereoDepth(4)] [error] Stereo alignment error: 1, trying to recover.
[component_container-2] [184430109159A30F00] [10.3.3.66] [12.142] [StereoDepth(4)] [error] Stereo rectification error: 1, trying to recover.
[component_container-2] [184430109159A30F00] [10.3.3.66] [49.335] [StereoDepth(4)] [error] Stereo alignment error: 2, trying to recover.
^C[WARNING] [launch]: user interrupted with ctrl-c (SIGINT)

As far as i know i should have an aligned depth topic avaliable, right? These are the ones i can see when doing ros2 topic list

root@victor-Legion-5-Pro-16ACH6H:/# ros2 topic list
/btAction
/joint_states
/oak/imu/data
/oak/nn/spatial_detections
/oak/rgb/camera_info
/oak/rgb/image_raw
/oak/rgb/image_raw/compressed
/oak/rgb/image_raw/compressedDepth
/oak/rgb/image_raw/theora
/oak/stereo/camera_info
/oak/stereo/image_raw
/oak/stereo/image_raw/compressed
/oak/stereo/image_raw/compressedDepth
/oak/stereo/image_raw/theora
/parameter_events
/robot_description
/rosout
/tf
/tf_static

Was this ever implemented for the OAK-D PRO W POE? I am running this in the dockerized setup btw.

[Serafadam]

Hi, aligned topic is available on /oak/stereo/image_raw, do you receive data on that one?

[Victorsoeby]

Hi @Serafadam, thank you for the quick reply.

Yes i am receiving data on both /oak/rgb/image_raw and /oak/stereo/image_raw. They may be alligned, but i was under the impression that they should share FOV, and thus thought that they were not aligned. According to Luxonis' website the FOV of both my depth and RGB sensor should be 150° / 127° / 79.5°.

As you can see in the below image, there is something up with the FOV, which i thought related to the Stereo alignment error and Stereo rectification error that the camera.launch.py file is throwing. The images side by side can be seen below, note my hand is not in the depth image.

Is this the intended behavior?

[Serafadam]

Hi, yes the FOV is different for those sensors, since they have different distortions & positions they might cover different areas, you could try rgb rectification (available in rgbd_pcl.launch` to check how that changes things. You can also try rgb depth alignment demo to see that alignment in effect better.

[Jenanaputra]

Hi @Serafadam , I got the same issue when trying to launch the camera.launch.py in my humble distro. The code then suddenly stop working with new error "[ERROR] [component_container-2]: process has died [pid 70472, exit code -11, cmd '/opt/ros/humble/lib/rclcpp_components/component_container --ros-args --log-level info --ros-args -r __node:=oak_container -r __ns:=/']."

For additional information, I used OAK-D POE in Ubuntu 22.04. Do you know how to overcome this issue ? I need to make sure that the camera.launch.py can work well in my laptop before trying to launch rtabmap.launch.py

Thanks!

[Serafadam]

Hi @Jenanaputra , sorry for the delay, could you share more information according to the BUG report template?

[Jenanaputra]

Sure @Serafadam

I am working with Ubuntu 22.04.1 LTS and ros2 humble. I installed the depthai-ros from source. There is no problem when I executed the stereo_inertial_node.launch.py by "ros2 launch depthai_examples stereo_inertial_node.launch.py".

Unfortunately, I got the error when trying to launch the camera.launch.py. The error and the log can be seen trough the following image.

[Serafadam]

Could you share more information as in bug report template? Running export DEPTHAI_DEBUG=1 before launching the example will print out more logs as well.