Closed mkarklins closed 1 year ago
Hi, to load custom YOLO model, you would need to set the "zoo" parameter to "path" and then provide the path to the .blob
file, similarly to how the current yolo example is set up.
Thank you very much - works as expected!
I just didn't know where to look for the problem based on the error log.
Describe the bug Unabe to launch custom spatial Yolo NN, however it works from plain pipeline without ROS.
The error message seems to be:
Minimal Reproducible Example
Here are the relevant files:
main.py - this is minimally adjusted segment.launch.py
``` import os from ament_index_python.packages import get_package_share_directory from launch import LaunchDescription from launch.actions import DeclareLaunchArgument, IncludeLaunchDescription, OpaqueFunction from launch.launch_description_sources import PythonLaunchDescriptionSource from launch.substitutions import LaunchConfiguration def launch_setup(context, *args, **kwargs): name = LaunchConfiguration('name').perform(context) depthai_prefix = get_package_share_directory("depthai_ros_driver") rviz_config = os.path.join(depthai_prefix, "config", "rviz", "segmentation.rviz") params_file = LaunchConfiguration("params_file") return [ IncludeLaunchDescription( PythonLaunchDescriptionSource( os.path.join(depthai_prefix, 'launch', 'camera.launch.py')), launch_arguments={"name": name, "params_file": params_file, "use_rviz": LaunchConfiguration("use_rviz"), "rviz_config": rviz_config}.items()) ] def generate_launch_description(): declared_arguments = [ DeclareLaunchArgument("name", default_value="oak"), DeclareLaunchArgument("params_file", default_value="./params.yaml"), DeclareLaunchArgument("use_rviz", default_value="True"), ] return LaunchDescription( declared_arguments + [OpaqueFunction(function=launch_setup)] ) ```params.yaml
``` /oak: ros__parameters: camera: i_nn_type: spatial i_pipeline_type: 'RGBD' nn: i_nn_config_path: 'nn/best_400.json' # i_nn_config_path: depthai_ros_driver/yolo ```best_400.json
``` { "model": { "xml": "best_400.xml", "bin": "best_400.bin" }, "nn_config": { "output_format": "detection", "NN_family": "YOLO", "input_size": "640x640", "NN_specific_metadata": { "classes": 1, "coordinates": 4, "anchors": [ 10.0, 13.0, 16.0, 30.0, 33.0, 23.0, 30.0, 61.0, 62.0, 45.0, 59.0, 119.0, 116.0, 90.0, 156.0, 198.0, 373.0, 326.0 ], "anchor_masks": { "side80": [ 0, 1, 2 ], "side40": [ 3, 4, 5 ], "side20": [ 6, 7, 8 ] }, "iou_threshold": 0.5, "confidence_threshold": 0.5 } }, "mappings": { "labels": [ "gates" ] }, "version": 1 } ```The NN exported from tools.luxonis.com - nn.zip
Executing with my custom model the pipeline fails:
ros2 launch main.py
, however if I change nn path to defaultdepthai_ros_driver/yolo
it works.I thought the problem would be with the model, however launching my custom model without ROS using the depthai-experiments repo from
depthai-experiments/gen2-yolo/device-decoding
usingpython3 main.py --config best_400.json
it works nicely.Expected behavior Expect the depthai_ros_driver node to launch successfully without errors.
Attach system log
non-verbose logs
``` mkarklins@mkarklins:~/code/temp$ ros2 launch main.py [INFO] [launch]: All log files can be found below /home/mkarklins/.ros/log/2023-11-09-08-19-54-082766-mkarklins-20082 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [rviz2-1]: process started with pid [20096] [INFO] [component_container-2]: process started with pid [20098] [rviz2-1] [INFO] [1699510794.592231944] [rviz2]: Stereo is NOT SUPPORTED [rviz2-1] [INFO] [1699510794.592396226] [rviz2]: OpenGl version: 4.5 (GLSL 4.5) [component_container-2] [INFO] [1699510794.593509562] [oak_container]: Load Library: /opt/ros/humble/lib/librobot_state_publisher_node.so [component_container-2] [INFO] [1699510794.595724030] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplatelog_system_information.py
``` { "architecture": "64bit ELF", "machine": "x86_64", "platform": "Linux-6.2.0-36-generic-x86_64-with-glibc2.35", "processor": "x86_64", "python_build": "main Jun 11 2023 05:26:28", "python_compiler": "GCC 11.4.0", "python_implementation": "CPython", "python_version": "3.10.12", "release": "6.2.0-36-generic", "system": "Linux", "version": "#37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct 9 15:34:04 UTC 2", "win32_ver": "", "packages": [ "action-msgs==1.2.1", "action-tutorials-interfaces==0.20.3", "action-tutorials-py==0.20.3", "actionlib-msgs==4.2.3", "actuator-msgs==0.0.1", "addict==2.4.0", "AHRS==0.3.1", "ament-cmake-google-benchmark==1.3.5", "ament-cmake-test==1.3.5", "ament-copyright==0.12.8", "ament-cppcheck==0.12.8", "ament-cpplint==0.12.8", "ament-flake8==0.12.8", "ament-index-python==1.4.0", "ament-lint==0.12.8", "ament-lint-cmake==0.12.8", "ament-package==0.14.0", "ament-pep257==0.12.8", "ament-uncrustify==0.12.8", "ament-xmllint==0.12.8", "angles==1.15.0", "ansi2html==1.8.0", "appdirs==1.4.4", "apturl==0.5.2", "argcomplete==3.1.1", "argparse==1.4.0", "asttokens==2.2.1", "attrs==23.1.0", "backcall==0.2.0", "beautifulsoup4==4.10.0", "beniget==0.4.1", "blinker==1.4", "blobconverter==1.4.2", "bloom==0.11.2", "bond==3.0.2", "boto3==1.28.80", "botocore==1.31.80", "breezy==3.2.1", "Brlapi==0.8.3", "Brotli==1.0.9", "builtin-interfaces==1.2.1", "camera-calibration==3.0.0", "cartographer-ros-msgs==2.0.9000", "catkin-pkg==1.0.0", "catkin-pkg-modules==1.0.0", "Cerberus==1.3.4", "certifi==2020.6.20", "chardet==4.0.0", "click==8.0.3", "colcon-argcomplete==0.3.3", "colcon-bash==0.5.0", "colcon-cd==0.1.1", "colcon-cmake==0.2.28", "colcon-common-extensions==0.3.0", "colcon-core==0.15.0", "colcon-defaults==0.2.8", "colcon-devtools==0.2.4", "colcon-installed-package-information==0.1.0", "colcon-library-path==0.2.1", "colcon-metadata==0.2.5", "colcon-notification==0.2.15", "colcon-output==0.2.13", "colcon-override-check==0.0.1", "colcon-package-information==0.3.3", "colcon-package-selection==0.2.10", "colcon-parallel-executor==0.3.0", "colcon-pkg-config==0.1.0", "colcon-powershell==0.4.0", "colcon-python-setup-py==0.2.8", "colcon-recursive-crawl==0.2.3", "colcon-ros==0.4.1", "colcon-test-result==0.3.8", "colcon-zsh==0.5.0", "colorama==0.4.4", "comm==0.1.3", "command-not-found==0.3", "composition-interfaces==1.2.1", "ConfigArgParse==1.5.5", "configobj==5.0.6", "contourpy==1.0.7", "cov-core==1.15.0", "coverage==7.2.7", "cryptography==3.4.8", "cupshelpers==1.0", "cv-bridge==3.2.1", "cycler==0.11.0", "dash==2.11.1", "dash-core-components==2.0.0", "dash-html-components==2.0.0", "dash-table==5.0.0", "dbus-python==1.2.18", "debugpy==1.6.7", "decorator==4.4.2", "defer==1.0.6", "demo-nodes-py==0.20.3", "Deprecated==1.2.13", "depthai==2.22.0.0", "depthai-pipeline-graph==0.0.5", "depthai-ros-driver==2.8.1", "depthai-ros-msgs==2.8.1", "depthai-sdk==1.13.1", "diagnostic-msgs==4.2.3", "diagnostic-updater==3.1.2", "distlib==0.3.4", "distro==1.7.0", "distro-info==1.1+ubuntu0.1", "docutils==0.17.1", "domain-coordinator==0.10.0", "dulwich==0.20.31", "dwb-msgs==1.1.12", "empy==3.3.4", "evdev==1.6.1", "example-interfaces==0.9.3", "examples-rclpy-executors==0.15.1", "examples-rclpy-minimal-action-client==0.15.1", "examples-rclpy-minimal-action-server==0.15.1", "examples-rclpy-minimal-client==0.15.1", "examples-rclpy-minimal-publisher==0.15.1", "examples-rclpy-minimal-service==0.15.1", "examples-rclpy-minimal-subscriber==0.15.1", "executing==1.2.0", "fastbencode==0.0.5", "fastimport==0.9.14", "fastjsonschema==2.18.0", "flake8==4.0.1", "Flask==2.2.5", "fonttools==4.40.0", "foxglove-msgs==2.2.0", "fs==2.4.12", "future==0.18.3", "gast==0.5.2", "gazebo-msgs==3.7.0", "geometry-msgs==4.2.3", "gpg==1.16.0", "gps-msgs==2.0.3", "html5lib==1.1", "httplib2==0.20.2", "idna==3.3", "image-geometry==3.2.1", "importlib-metadata==4.6.4", "importlib-resources==5.12.0", "iniconfig==1.1.1", "inotify-simple==1.3.5", "interactive-markers==2.3.2", "ipykernel==6.24.0", "ipython==8.14.0", "ipywidgets==8.0.7", "itsdangerous==2.1.2", "jedi==0.18.2", "jeepney==0.7.1", "Jinja2==3.1.2", "jmespath==1.0.1", "joblib==1.3.1", "jsonschema==4.17.3", "jupyter_client==8.3.0", "jupyter_core==5.3.1", "jupyterlab-widgets==3.0.8", "kconfiglib==14.1.0", "keyring==23.5.0", "kiwisolver==1.4.4", "language-selector==0.1", "lark==1.1.1", "laser-geometry==2.4.0", "launch==1.0.4", "launch-ros==0.19.6", "launch-testing==1.0.4", "launch-testing-ros==0.19.6", "launch-xml==1.0.4", "launch-yaml==1.0.4", "launchpadlib==1.10.16", "lazr.restfulclient==0.14.4", "lazr.uri==1.0.6", "lifecycle-msgs==1.2.1", "logging-demo==0.20.3", "louis==3.20.0", "lxml==4.9.2", "lz4==3.1.3+dfsg", "macaroonbakery==1.3.1", "map-msgs==2.1.0", "MarkupSafe==2.1.3", "marshmallow==3.17.0", "matplotlib==3.7.1", "matplotlib-inline==0.1.6", "mccabe==0.6.1", "mercurial==6.1.1", "message-filters==4.3.3", "more-itertools==8.10.0", "mpi4py==3.1.3", "mpmath==1.3.0", "nav-2d-msgs==1.1.12", "nav-msgs==4.2.3", "nav2-common==1.1.12", "nav2-msgs==1.1.12", "nav2-simple-commander==1.0.0", "nbformat==5.7.0", "nest-asyncio==1.5.6", "netifaces==0.11.0", "nose2==0.9.2", "notify2==0.3", "numpy==1.24.3", "nunavut==2.1.0", "oauthlib==3.2.0", "octomap-msgs==2.0.0", "olefile==0.46", "open3d==0.17.0", "opencv-contrib-python==4.8.1.78", "opencv-python==4.8.0.76", "osrf-pycommon==2.0.2", "packaging==23.1", "pandas==2.0.2", "parso==0.8.3", "patiencediff==0.2.1", "pcl-msgs==1.0.0", "pendulum-msgs==0.20.3", "pexpect==4.8.0", "pickleshare==0.7.5", "Pillow==10.0.0", "pip==23.3.1", "pkgconfig==1.5.5", "platformdirs==3.9.1", "plotly==5.15.0", "pluggy==0.13.0", "ply==3.11", "prompt-toolkit==3.0.39", "protobuf==3.12.4", "psutil==5.9.5", "ptyprocess==0.7.0", "pure-eval==0.2.2", "py==1.10.0", "pybind11==2.9.1", "pycairo==1.20.1", "pycodestyle==2.8.0", "pycups==2.0.1", "pydocstyle==6.1.1", "pydot==1.4.2", "pydsdl==1.20.0", "pyflakes==2.4.0", "pygame==2.5.1", "PyGithub==1.55", "Pygments==2.11.2", "PyGObject==3.42.1", "PyJWT==2.3.0", "pymacaroons==0.13.0", "pymavlink==2.4.39", "PyNaCl==1.5.0", "PyOpenGL==3.1.7", "pyparsing==2.4.7", "PyQt5==5.15.6", "PyQt5-sip==12.9.1", "pyquaternion==0.9.9", "pyRFC3339==1.1", "pyros-genmsg==0.5.8", "pyrsistent==0.19.3", "pyserial==3.5", "pytest==6.2.5", "pytest-cov==3.0.0", "python-apt==2.4.0+ubuntu2", "python-dateutil==2.8.2", "python-debian==0.1.43+ubuntu1.1", "python-gitlab==2.10.1", "python-qt-binding==1.1.1", "python-xlib==0.33", "pythran==0.10.0", "pytube==15.0.0", "PyTurboJPEG==1.6.4", "pytz==2022.1", "pyulog==1.0.1", "pyxdg==0.27", "PyYAML==5.4.1", "pyzmq==25.1.0", "qt-dotgraph==2.2.2", "qt-gui==2.2.2", "qt-gui-cpp==2.2.2", "qt-gui-py-common==2.2.2", "Qt.py==1.3.8", "quality-of-service-demo-py==0.20.3", "rcl-interfaces==1.2.1", "rclpy==3.3.10", "rcutils==5.1.3", "realsense2-camera-msgs==4.54.1", "reportlab==3.6.8", "requests==2.25.1", "requests-toolbelt==0.9.1", "resource-retriever==3.1.1", "retrying==1.3.4", "rmw-dds-common==1.6.0", "roman==3.3", "ros2action==0.18.7", "ros2bag==0.15.8", "ros2cli==0.18.7", "ros2component==0.18.7", "ros2doctor==0.18.7", "ros2interface==0.18.7", "ros2launch==0.19.6", "ros2lifecycle==0.18.7", "ros2multicast==0.18.7", "ros2node==0.18.7", "ros2param==0.18.7", "ros2pkg==0.18.7", "ros2run==0.18.7", "ros2service==0.18.7", "ros2topic==0.18.7", "rosbag2-interfaces==0.15.8", "rosbag2-py==0.15.8", "rosdep==0.22.2", "rosdep-modules==0.22.2", "rosdistro==0.9.0", "rosdistro-modules==0.9.0", "rosgraph-msgs==1.2.1", "rosidl-adapter==3.1.5", "rosidl-cli==3.1.5", "rosidl-cmake==3.1.5", "rosidl-generator-c==3.1.5", "rosidl-generator-cpp==3.1.5", "rosidl-generator-py==0.14.4", "rosidl-parser==3.1.5", "rosidl-runtime-py==0.9.3", "rosidl-typesupport-c==2.0.1", "rosidl-typesupport-cpp==2.0.1", "rosidl-typesupport-fastrtps-c==2.2.1", "rosidl-typesupport-fastrtps-cpp==2.2.1", "rosidl-typesupport-introspection-c==3.1.5", "rosidl-typesupport-introspection-cpp==3.1.5", "rospkg-modules==1.5.0", "rpyutils==0.2.1", "rqt-action==2.0.1", "rqt-bag==1.1.4", "rqt-bag-plugins==1.1.4", "rqt-console==2.0.2", "rqt-graph==1.3.0", "rqt-gui==1.1.5", "rqt-gui-py==1.1.5", "rqt-msg==1.2.0", "rqt-plot==1.1.2", "rqt-publisher==1.5.0", "rqt-py-common==1.1.5", "rqt-py-console==1.0.2", "rqt-reconfigure==1.1.1", "rqt-service-caller==1.0.5", "rqt-shell==1.0.2", "rqt-srv==1.0.3", "rqt-topic==1.5.0", "rtabmap-msgs==0.21.1", "rtabmap-python==0.1.0", "s3transfer==0.7.0", "scikit-learn==1.3.0", "scipy==1.8.0", "SecretStorage==3.3.1", "sensor-msgs==4.2.3", "sensor-msgs-py==4.2.3", "sentry-sdk==1.21.0", "setuptools==59.6.0", "shape-msgs==4.2.3", "six==1.16.0", "slam-toolbox==2.6.6", "smclib==3.0.2", "snowballstemmer==2.2.0", "soupsieve==2.3.1", "sros2==0.10.4", "stack-data==0.6.2", "statistics-msgs==1.2.1", "std-msgs==4.2.3", "std-srvs==4.2.3", "stereo-msgs==4.2.3", "sympy==1.12", "systemd-python==234", "teleop-twist-keyboard==2.3.2", "tenacity==8.2.2", "tf2-geometry-msgs==0.25.4", "tf2-kdl==0.25.4", "tf2-msgs==0.25.4", "tf2-py==0.25.4", "tf2-ros-py==0.25.4", "tf2-tools==0.25.4", "theora-image-transport==2.5.0", "threadpoolctl==3.2.0", "toml==0.10.2", "topic-monitor==0.20.3", "tornado==6.3.2", "tqdm==4.65.0", "traitlets==5.9.0", "trajectory-msgs==4.2.3", "turtlebot3-example==2.1.5", "turtlebot3-msgs==2.2.3", "turtlebot3-teleop==2.1.5", "turtlesim==1.4.2", "types-pyside2==5.15.2.1.6", "typing_extensions==4.7.1", "tzdata==2023.3", "ubuntu-advantage-tools==8001", "ubuntu-drivers-common==0.0.0", "ufoLib2==0.13.1", "ufw==0.36.1", "unattended-upgrades==0.1", "unicodedata2==14.0.0", "unique-identifier-msgs==2.2.1", "urllib3==2.0.7", "vcstool==0.3.0", "vcstools==0.1.42", "velodyne-msgs==2.4.0", "vision-msgs==4.1.0", "visualization-msgs==4.2.3", "wadllib==1.3.6", "wcwidth==0.2.6", "webencodings==0.5.1", "Werkzeug==2.2.3", "wheel==0.37.1", "widgetsnbextension==4.0.8", "wrapt==1.13.3", "xacro==2.0.8", "xdg==5", "xkeysnail @ file:///home/mkarklins/Downloads/kinto-master/xkeysnail", "xkit==0.0.0", "xmltodict==0.13.0", "zipp==1.0.0" ], "usb": [ "NoLib" ], "uname": [ "Linux mkarklins 6.2.0-36-generic #37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct 9 15:34:04 UTC 2 x86_64 x86_64" ] } System info gathered successfully - saved as "log_system_information.json" ```Installed packages
``` Package: ros-humble-depthai Version: 2.22.0-1jammy.20230623.022440 Priority: optional Section: misc Maintainer: Sachin Guruswamy