Cannot colcon build depthai-ros on Jetson Orin Jetpack 6

[brianlmerritt]

Check if issue already exists

https://github.com/luxonis/depthai-ros/issues/419 mentions the issue on Xavier, but has not progressed

Describe the bug Standard Jetpack 6 Ubuntu 22.04 build with ROS 2 humble compiles ok with Orbbec Astra but not with DepthAI See below

Starting >>> depthai_examples
Finished <<< depthai_examples [1.06s]                                                                       
Starting >>> depthai_ros_driver
Finished <<< depthai_ros_driver [3.89s]                                                                       
[Processing: depthai_filters]                               
[Processing: depthai_filters]                                        
--- stderr: depthai_filters                                           
In file included from /home/blm/limo_weston_ws/src/depthai-ros/depthai_filters/src/wls_filter.cpp:1:
/home/blm/limo_weston_ws/src/depthai-ros/depthai_filters/include/depthai_filters/wls_filter.hpp:8:10: fatal error: opencv2/ximgproc/disparity_filter.hpp: No such file or directory
    8 | #include "opencv2/ximgproc/disparity_filter.hpp"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
gmake[2]: *** [CMakeFiles/depthai_filters.dir/build.make:118: CMakeFiles/depthai_filters.dir/src/wls_filter.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
gmake[1]: *** [CMakeFiles/Makefile2:137: CMakeFiles/depthai_filters.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2
---
Failed   <<< depthai_filters [1min 27s, exited with code 2]

Summary: 13 packages finished [1min 34s]
  1 package failed: depthai_filters

Minimal Reproducible Example

Install Jetpack 6 on Jetson (must be Orin + for Jetpack 6 Install ROS 2 humble, install depthai-sdk and ros-humble-depthai Git clone depthai-ros in src and colcon build --symlink-install reproduces the above error.

If available launch files don't work in your case, please check if you also get errors while running: not applicable

Expected behavior colcon build should work or the necessary dependencies should be included (or workarounds mentioned) in depthai-ros

Screenshots If applicable, add screenshots to help explain your problem.

Pipeline Graph

Not applicable

Attach system log

• Provide output of log_system_information.py -

python3 log_system_information.py 
{
    "architecture": "64bit ELF",
    "machine": "aarch64",
    "platform": "Linux-5.15.122-tegra-aarch64-with-glibc2.35",
    "processor": "aarch64",
    "python_build": "main Nov 20 2023 15:14:05",
    "python_compiler": "GCC 11.4.0",
    "python_implementation": "CPython",
    "python_version": "3.10.12",
    "release": "5.15.122-tegra",
    "system": "Linux",
    "version": "#1 SMP PREEMPT Mon Dec 18 21:24:25 PST 2023",
    "win32_ver": "",
    "packages": [
        "action-msgs==1.2.1",
        "action-tutorials-interfaces==0.20.3",
        "action-tutorials-py==0.20.3",
        "actionlib-msgs==4.2.3",
        "ament-cmake-test==1.3.7",
        "ament-copyright==0.12.10",
        "ament-cppcheck==0.12.10",
        "ament-cpplint==0.12.10",
        "ament-flake8==0.12.10",
        "ament-index-python==1.4.0",
        "ament-lint==0.12.10",
        "ament-lint-cmake==0.12.10",
        "ament-package==0.14.0",
        "ament-pep257==0.12.10",
        "ament-uncrustify==0.12.10",
        "ament-xmllint==0.12.10",
        "angles==1.15.0",
        "appdirs==1.4.4",
        "apturl==0.5.2",
        "argcomplete==1.8.1",
        "attrs==21.2.0",
        "bcrypt==3.2.0",
        "beniget==0.4.1",
        "blinker==1.4",
        "bloom==0.11.2",
        "bond==3.0.2",
        "breezy==3.2.1",
        "Brlapi==0.8.3",
        "Brotli==1.0.9",
        "builtin-interfaces==1.2.1",
        "camera-calibration==3.0.0",
        "cartographer-ros-msgs==2.0.9000",
        "catkin-pkg==1.0.0",
        "catkin-pkg-modules==1.0.0",
        "certifi==2020.6.20",
        "chardet==4.0.0",
        "click==8.0.3",
        "colcon-argcomplete==0.3.3",
        "colcon-bash==0.5.0",
        "colcon-cd==0.1.1",
        "colcon-cmake==0.2.28",
        "colcon-common-extensions==0.3.0",
        "colcon-core==0.15.1",
        "colcon-defaults==0.2.8",
        "colcon-devtools==0.2.4",
        "colcon-installed-package-information==0.1.0",
        "colcon-library-path==0.2.1",
        "colcon-metadata==0.2.5",
        "colcon-notification==0.2.15",
        "colcon-output==0.2.13",
        "colcon-override-check==0.0.1",
        "colcon-package-information==0.3.3",
        "colcon-package-selection==0.2.10",
        "colcon-parallel-executor==0.3.0",
        "colcon-pkg-config==0.1.0",
        "colcon-powershell==0.4.0",
        "colcon-python-setup-py==0.2.8",
        "colcon-recursive-crawl==0.2.3",
        "colcon-ros==0.4.1",
        "colcon-test-result==0.3.8",
        "colcon-zsh==0.5.0",
        "colorama==0.4.4",
        "composition-interfaces==1.2.1",
        "configobj==5.0.6",
        "cov-core==1.15.0",
        "coverage==6.2",
        "cryptography==3.4.8",
        "cupshelpers==1.0",
        "cv-bridge==3.2.1",
        "cycler==0.11.0",
        "dbus-python==1.2.18",
        "decorator==4.4.2",
        "defer==1.0.6",
        "demo-nodes-py==0.20.3",
        "Deprecated==1.2.13",
        "depthai==2.24.0.0",
        "diagnostic-msgs==4.2.3",
        "diagnostic-updater==3.1.2",
        "distlib==0.3.4",
        "distro==1.7.0",
        "distro-info==1.1+ubuntu0.2",
        "DockerHub-API @ git+https://github.com/Granulate/DockerHub-API.git@27015dbd00c0e0550dd34287db4b70625985edc3",
        "docutils==0.17.1",
        "domain-coordinator==0.10.0",
        "dulwich==0.20.31",
        "duplicity==0.8.21",
        "dwb-msgs==1.1.12",
        "empy==3.3.4",
        "example-interfaces==0.9.3",
        "examples-rclpy-executors==0.15.1",
        "examples-rclpy-minimal-action-client==0.15.1",
        "examples-rclpy-minimal-action-server==0.15.1",
        "examples-rclpy-minimal-client==0.15.1",
        "examples-rclpy-minimal-publisher==0.15.1",
        "examples-rclpy-minimal-service==0.15.1",
        "examples-rclpy-minimal-subscriber==0.15.1",
        "fastbencode==0.0.5",
        "fasteners==0.14.1",
        "fastimport==0.9.14",
        "flake8==4.0.1",
        "fonttools==4.29.1",
        "foxglove-msgs==2.3.0",
        "fs==2.4.12",
        "furl==2.1.3",
        "future==0.18.2",
        "gast==0.5.2",
        "geometry-msgs==4.2.3",
        "gpg==1.16.0",
        "graphsurgeon==0.4.6",
        "httplib2==0.20.2",
        "idna==3.3",
        "image-geometry==3.2.1",
        "importlib-metadata==4.6.4",
        "iniconfig==1.1.1",
        "interactive-markers==2.3.2",
        "jeepney==0.7.1",
        "Jetson.GPIO==2.1.3",
        "joint-state-publisher==2.4.0",
        "joint-state-publisher-gui==2.4.0",
        "keyring==23.5.0",
        "kiwisolver==1.3.2",
        "language-selector==0.1",
        "lark==1.1.1",
        "laser-geometry==2.4.0",
        "launch==1.0.4",
        "launch-ros==0.19.7",
        "launch-testing==1.0.4",
        "launch-testing-ros==0.19.7",
        "launch-xml==1.0.4",
        "launch-yaml==1.0.4",
        "launchpadlib==1.10.16",
        "lazr.restfulclient==0.14.4",
        "lazr.uri==1.0.6",
        "lifecycle-msgs==1.2.1",
        "lockfile==0.12.2",
        "logging-demo==0.20.3",
        "louis==3.20.0",
        "lxml==4.8.0",
        "lz4==3.1.3+dfsg",
        "macaroonbakery==1.3.1",
        "Mako==1.1.3",
        "map-msgs==2.1.0",
        "MarkupSafe==2.0.1",
        "matplotlib==3.5.1",
        "mccabe==0.6.1",
        "mercurial==6.1.1",
        "message-filters==4.3.3",
        "monotonic==1.6",
        "more-itertools==8.10.0",
        "mpi4py==3.1.3",
        "mpmath==0.0.0",
        "nav-2d-msgs==1.1.12",
        "nav-msgs==4.2.3",
        "nav2-common==1.1.12",
        "nav2-msgs==1.1.12",
        "nav2-simple-commander==1.0.0",
        "netifaces==0.11.0",
        "nose2==0.9.2",
        "notify2==0.3",
        "numpy==1.21.5",
        "oauthlib==3.2.0",
        "olefile==0.46",
        "onboard==1.4.1",
        "onnx-graphsurgeon==0.3.12",
        "opencv-contrib-python==4.9.0.80",
        "orderedmultidict==1.0.1",
        "osrf-pycommon==2.0.2",
        "packaging==21.3",
        "pandas==1.3.5",
        "paramiko==2.9.3",
        "patiencediff==0.2.1",
        "pcl-msgs==1.0.0",
        "pendulum-msgs==0.20.3",
        "pexpect==4.8.0",
        "Pillow==9.0.1",
        "pip==24.0",
        "pluggy==0.13.0",
        "ply==3.11",
        "protobuf==3.12.4",
        "psutil==5.9.0",
        "ptyprocess==0.7.0",
        "py==1.10.0",
        "pycairo==1.20.1",
        "pycodestyle==2.8.0",
        "pycups==2.0.1",
        "pydocstyle==6.1.1",
        "pydot==1.4.2",
        "pyflakes==2.4.0",
        "PyGithub==1.55",
        "Pygments==2.11.2",
        "PyGObject==3.42.1",
        "PyJWT==2.3.0",
        "pylimo==0.3.5",
        "pymacaroons==0.13.0",
        "PyNaCl==1.5.0",
        "PyOpenGL==3.1.5",
        "pyparsing==2.4.7",
        "PyQt5==5.15.6",
        "PyQt5-sip==12.9.1",
        "pyRFC3339==1.1",
        "pyserial==3.5",
        "pytest==6.2.5",
        "pytest-cov==3.0.0",
        "python-apt==2.4.0+ubuntu2",
        "python-dateutil==2.8.1",
        "python-dbusmock==0.27.5",
        "python-debian==0.1.43+ubuntu1.1",
        "python-gitlab==2.10.1",
        "python-qt-binding==1.1.2",
        "pythran==0.10.0",
        "pytz==2022.1",
        "pyxdg==0.27",
        "PyYAML==5.4.1",
        "qt-dotgraph==2.2.3",
        "qt-gui==2.2.3",
        "qt-gui-cpp==2.2.3",
        "qt-gui-py-common==2.2.3",
        "quality-of-service-demo-py==0.20.3",
        "rcl-interfaces==1.2.1",
        "rclpy==3.3.11",
        "rcutils==5.1.4",
        "requests==2.25.1",
        "requests-toolbelt==0.9.1",
        "resource-retriever==3.1.1",
        "rmw-dds-common==1.6.0",
        "roman==3.3",
        "ros2action==0.18.8",
        "ros2bag==0.15.9",
        "ros2cli==0.18.8",
        "ros2component==0.18.8",
        "ros2doctor==0.18.8",
        "ros2interface==0.18.8",
        "ros2launch==0.19.7",
        "ros2lifecycle==0.18.8",
        "ros2multicast==0.18.8",
        "ros2node==0.18.8",
        "ros2param==0.18.8",
        "ros2pkg==0.18.8",
        "ros2run==0.18.8",
        "ros2service==0.18.8",
        "ros2topic==0.18.8",
        "rosbag2-interfaces==0.15.9",
        "rosbag2-py==0.15.9",
        "rosdep==0.22.2",
        "rosdep-modules==0.22.2",
        "rosdistro==0.9.0",
        "rosdistro-modules==0.9.0",
        "rosgraph-msgs==1.2.1",
        "rosidl-adapter==3.1.5",
        "rosidl-cli==3.1.5",
        "rosidl-cmake==3.1.5",
        "rosidl-generator-c==3.1.5",
        "rosidl-generator-cpp==3.1.5",
        "rosidl-generator-py==0.14.4",
        "rosidl-parser==3.1.5",
        "rosidl-runtime-py==0.9.3",
        "rosidl-typesupport-c==2.0.1",
        "rosidl-typesupport-cpp==2.0.1",
        "rosidl-typesupport-fastrtps-c==2.2.2",
        "rosidl-typesupport-fastrtps-cpp==2.2.2",
        "rosidl-typesupport-introspection-c==3.1.5",
        "rosidl-typesupport-introspection-cpp==3.1.5",
        "rospkg-modules==1.5.0",
        "rpyutils==0.2.1",
        "rqt-action==2.0.1",
        "rqt-bag==1.1.4",
        "rqt-bag-plugins==1.1.4",
        "rqt-console==2.0.2",
        "rqt-graph==1.3.0",
        "rqt-gui==1.1.6",
        "rqt-gui-py==1.1.6",
        "rqt-msg==1.2.0",
        "rqt-plot==1.1.2",
        "rqt-publisher==1.5.0",
        "rqt-py-common==1.1.6",
        "rqt-py-console==1.0.2",
        "rqt-reconfigure==1.1.2",
        "rqt-service-caller==1.0.5",
        "rqt-shell==1.0.2",
        "rqt-srv==1.0.3",
        "rqt-topic==1.5.0",
        "scipy==1.8.0",
        "SecretStorage==3.3.1",
        "sensor-msgs==4.2.3",
        "sensor-msgs-py==4.2.3",
        "setuptools==59.6.0",
        "shape-msgs==4.2.3",
        "six==1.16.0",
        "slam-toolbox==2.6.8",
        "smclib==3.0.2",
        "snowballstemmer==2.2.0",
        "sros2==0.10.4",
        "statistics-msgs==1.2.1",
        "std-msgs==4.2.3",
        "std-srvs==4.2.3",
        "stereo-msgs==4.2.3",
        "sympy==1.9",
        "systemd-python==234",
        "teleop-twist-keyboard==2.3.2",
        "tensorrt==8.6.2",
        "tensorrt-dispatch==8.6.2",
        "tensorrt-lean==8.6.2",
        "tf2-geometry-msgs==0.25.5",
        "tf2-kdl==0.25.5",
        "tf2-msgs==0.25.5",
        "tf2-py==0.25.5",
        "tf2-ros-py==0.25.5",
        "tf2-tools==0.25.5",
        "theora-image-transport==2.5.0",
        "toml==0.10.2",
        "topic-monitor==0.20.3",
        "trajectory-msgs==4.2.3",
        "turtlesim==1.4.2",
        "ubuntu-advantage-tools==8001",
        "ubuntu-drivers-common==0.0.0",
        "uff==0.6.9",
        "ufoLib2==0.13.1",
        "unicodedata2==14.0.0",
        "unique-identifier-msgs==2.2.1",
        "urllib3==1.26.5",
        "urwid==2.1.2",
        "vcstool==0.3.0",
        "vcstools==0.1.42",
        "vision-msgs==4.1.1",
        "visualization-msgs==4.2.3",
        "wadllib==1.3.6",
        "wget==3.2",
        "wheel==0.37.1",
        "wrapt==1.13.3",
        "xacro==2.0.8",
        "xdg==5",
        "xkit==0.0.0",
        "zipp==1.0.0"
    ],
    "usb": [
        "NoLib"
    ],
    "uname": [
        "Linux ubuntu 5.15.122-tegra #1 SMP PREEMPT Mon Dec 18 21:24:25 PST 2023 aarch64 aarch64"
    ]
}

• Which OS/OS version are you using? UBUNTU 22.04
• Which ROS version are you using? ROS 2
• Which ROS distribution are you using ? Humble
• Is depthai-ros built from source or installed from apt - APT but either makes no difference
• Is depthai/depthai-core library installed from rosdep or manually? For rosdep install, check if ros-<rosdistro>-depthai package is installed, manual install can be checked with ldconfig -p | grep depthai- Don't know
• Please include versions of following packages - apt show ros-$ROS_DISTRO-depthai ros-$ROS_DISTRO-depthai-ros ros-$ROS_DISTRO-depthai-bridge ros-$ROS_DISTRO-depthai-ros-msgs ros-$ROS_DISTRO-depthai-ros-driver
• To get additional logs, set DEPTHAI_DEBUG=1 and paste the logs, either from command line or from latest log in ~/.ros/log Not applicable as it doesn't compile

Additional context I have seen suggestions go to https://github.com/mdegans/nano_build_opencv/ but even after modifying the code to work with Ubuntu 22.04 and install opencv 4.9 this still fails because the build does not create the necessary headers for opencv2/ximgproc I used locate *opencv*/*.h to look for all related files and all I got was duplicates of what was already installed with the nvidia-opencv-dev so it isn't just python or cmake looking at the wrong files.

If there is an opencv build that includes the necessary files, happy to try it, or any work around.

[brianlmerritt]

Just noticed I should have been going for

locate opencv/*.hpp

[brianlmerritt]

Just in case someone finds this closed ticket, first I updated nano_build_opencv for jetpack 6 to reinstall opencv 4.9 from source.

https://github.com/brianlmerritt/nano_build_opencv

I then created a separate workspace depthai_ws/src/ with depthai-ros there, and ran the build script in the top ws folder. I was trying to build inside the agilex limo ws but had to run build.sh to get it to work.