search

luxonis / depthai-ros

Official ROS Driver for DepthAI Sensors.
MIT License
261 stars 191 forks source link

camera.launch: IMU Data is (sometimes) not being published #287

Open everskies opened 1 year ago

everskies commented 1 year ago

Describe the bug I've been trying to get the camera.launch.py example to publish IMU data together with the stereo data and RGB image. The /oak/imu/data topic gets created:

ros2 topic list
/joint_states
/oak/imu/data
/oak/rgb/camera_info
/oak/rgb/image_raw
/oak/rgb/image_raw/compressed
/oak/rgb/image_raw/compressedDepth
/oak/rgb/image_raw/theora
/oak/stereo/camera_info
/oak/stereo/image_raw
/oak/stereo/image_raw/compressed
/oak/stereo/image_raw/compressedDepth
/oak/stereo/image_raw/theora
/parameter_events
/robot_description
/rosout
/tf
/tf_static

However, no data seems to arrive:

ros2 topic hz /oak/imu/data
WARNING: topic [/oak/imu/data] does not appear to be published yet

I've tried both the current humble release and one built from scratch.

Minimal Reproducible Example ros2 launch depthai_ros_driver camera.launch.py

[INFO] [launch]: All log files can be found below /home/sleziak/.ros/log/2023-04-18-22-47-30-727616-bubble-2551
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [robot_state_publisher-1]: process started with pid [2562]
[INFO] [component_container-2]: process started with pid [2564]
[component_container-2] [INFO] [1681850852.374532197] [oak_container]: Load Library: /home/sleziak/ros/install/depthai_ros_driver/lib/libdepthai_ros_driver.so
[robot_state_publisher-1] [INFO] [1681850852.540030735] [oak_state_publisher]: got segment oak
[robot_state_publisher-1] [INFO] [1681850852.540405935] [oak_state_publisher]: got segment oak-d-base-frame
[robot_state_publisher-1] [INFO] [1681850852.540491657] [oak_state_publisher]: got segment oak_imu_frame
[robot_state_publisher-1] [INFO] [1681850852.540537342] [oak_state_publisher]: got segment oak_left_camera_frame
[robot_state_publisher-1] [INFO] [1681850852.540575471] [oak_state_publisher]: got segment oak_left_camera_optical_frame
[robot_state_publisher-1] [INFO] [1681850852.540611785] [oak_state_publisher]: got segment oak_model_origin
[robot_state_publisher-1] [INFO] [1681850852.540646452] [oak_state_publisher]: got segment oak_rgb_camera_frame
[robot_state_publisher-1] [INFO] [1681850852.540679822] [oak_state_publisher]: got segment oak_rgb_camera_optical_frame
[robot_state_publisher-1] [INFO] [1681850852.540713359] [oak_state_publisher]: got segment oak_right_camera_frame
[robot_state_publisher-1] [INFO] [1681850852.540746488] [oak_state_publisher]: got segment oak_right_camera_optical_frame
[component_container-2] [INFO] [1681850852.691349716] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate<depthai_ros_driver::Camera>
[component_container-2] [INFO] [1681850852.691494308] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<depthai_ros_driver::Camera>
[component_container-2] [INFO] [1681850852.745608206] [oak]: No ip/mxid specified, connecting to the next available device.
[component_container-2] [INFO] [1681850855.725776186] [oak]: Camera with MXID: 18443010F1213B1300 and Name: 1.1.2 connected!
[component_container-2] [INFO] [1681850855.727304153] [oak]: USB SPEED: SUPER
[component_container-2] [INFO] [1681850855.750146502] [oak]: Device type: OAK-D-PRO-AF
[component_container-2] [INFO] [1681850855.757216437] [oak]: Pipeline type: RGBD
[component_container-2] [INFO] [1681850855.801349463] [oak]: NN Family: mobilenet
[component_container-2] [INFO] [1681850856.071826360] [oak]: NN input size: 300 x 300. Resizing input image in case of different dimensions.
[component_container-2] [INFO] [1681850856.277071831] [oak]: Finished setting up pipeline.
[component_container-2] [INFO] [1681850857.683827872] [oak]: Camera ready!
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/oak' in container '/oak_container'

Expected behavior There should be data being published to the /oak/imu/data topic

log_system_information.json ``` { "architecture": "64bit ELF", "machine": "aarch64", "platform": "Linux-5.15.0-1026-raspi-aarch64-with-glibc2.35", "processor": "aarch64", "python_build": "main Mar 10 2023 10:55:28", "python_compiler": "GCC 11.3.0", "python_implementation": "CPython", "python_version": "3.10.6", "release": "5.15.0-1026-raspi", "system": "Linux", "version": "#28-Ubuntu SMP PREEMPT Fri Mar 10 14:28:52 UTC 2023", "win32_ver": "", "packages": [ "action-msgs==1.2.1", "action-tutorials-interfaces==0.20.3", "action-tutorials-py==0.20.3", "actionlib-msgs==4.2.3", "ament-cmake-test==1.3.3", "ament-copyright==0.12.5", "ament-cppcheck==0.12.5", "ament-cpplint==0.12.5", "ament-flake8==0.12.5", "ament-index-python==1.4.0", "ament-lint==0.12.5", "ament-lint-cmake==0.12.5", "ament-package==0.14.0", "ament-pep257==0.12.5", "ament-uncrustify==0.12.5", "ament-xmllint==0.12.5", "angles==1.15.0", "appdirs==1.4.4", "argcomplete==1.8.1", "attrs==21.2.0", "Automat==20.2.0", "Babel==2.8.0", "bcrypt==3.2.0", "beautifulsoup4==4.10.0", "beniget==0.4.1", "blinker==1.4", "bond==3.0.2", "Brotli==1.0.9", "builtin-interfaces==1.2.1", "camera-calibration==3.0.0", "catkin-pkg==0.5.2", "catkin-pkg-modules==0.5.2", "certifi==2020.6.20", "chardet==4.0.0", "charset-normalizer==3.1.0", "click==8.0.3", "cloud-init==23.1.1", "colcon-argcomplete==0.3.3", "colcon-bash==0.4.2", "colcon-cd==0.1.1", "colcon-cmake==0.2.27", "colcon-common-extensions==0.3.0", "colcon-core==0.12.1", "colcon-defaults==0.2.8", "colcon-devtools==0.2.3", "colcon-installed-package-information==0.1.0", "colcon-library-path==0.2.1", "colcon-metadata==0.2.5", "colcon-notification==0.2.15", "colcon-output==0.2.13", "colcon-override-check==0.0.1", "colcon-package-information==0.3.3", "colcon-package-selection==0.2.10", "colcon-parallel-executor==0.2.4", "colcon-pkg-config==0.1.0", "colcon-powershell==0.3.7", "colcon-python-setup-py==0.2.8", "colcon-recursive-crawl==0.2.1", "colcon-ros==0.3.23", "colcon-test-result==0.3.8", "colcon-zsh==0.4.0", "colorama==0.4.4", "command-not-found==0.3", "composition-interfaces==1.2.1", "configobj==5.0.6", "constantly==15.1.0", "cov-core==1.15.0", "coverage==6.2", "cryptography==3.4.8", "cv-bridge==3.2.1", "cycler==0.11.0", "dbus-python==1.2.18", "decorator==4.4.2", "demo-nodes-py==0.20.3", "depthai==2.21.2.0", "depthai-ros-driver==2.7.1", "depthai-ros-msgs==2.7.1", "diagnostic-msgs==4.2.3", "diagnostic-updater==3.1.2", "distlib==0.3.6", "distro==1.7.0", "distro-info===1.1build1", "docutils==0.17.1", "domain-coordinator==0.10.0", "dwb-msgs==1.1.6", "empy==3.3.4", "example-interfaces==0.9.3", "examples-rclpy-executors==0.15.1", "examples-rclpy-minimal-action-client==0.15.1", "examples-rclpy-minimal-action-server==0.15.1", "examples-rclpy-minimal-client==0.15.1", "examples-rclpy-minimal-publisher==0.15.1", "examples-rclpy-minimal-service==0.15.1", "examples-rclpy-minimal-subscriber==0.15.1", "filelock==3.9.0", "flake8==4.0.1", "fonttools==4.29.1", "foxglove-msgs==2.1.1", "fs==2.4.12", "gast==0.5.2", "geographic-msgs==1.0.4", "geometry-msgs==4.2.3", "html5lib==1.1", "httplib2==0.20.2", "hyperlink==21.0.0", "idna==3.3", "image-geometry==3.2.1", "importlib-metadata==4.6.4", "incremental==21.3.0", "iniconfig==1.1.1", "interactive-markers==2.3.2", "jeepney==0.7.1", "Jinja2==3.0.3", "joint-state-publisher==2.3.0", "jsonpatch==1.32", "jsonpointer==2.0", "jsonschema==3.2.0", "keyring==23.5.0", "kiwisolver==1.3.2", "lark==1.1.1", "laser-geometry==2.4.0", "launch==1.0.4", "launch-ros==0.19.4", "launch-testing==1.0.4", "launch-testing-ros==0.19.4", "launch-xml==1.0.4", "launch-yaml==1.0.4", "launchpadlib==1.10.16", "lazr.restfulclient==0.14.4", "lazr.uri==1.0.6", "lifecycle-msgs==1.2.1", "logging-demo==0.20.3", "lxml==4.8.0", "lz4==3.1.3+dfsg", "map-msgs==2.1.0", "MarkupSafe==2.0.1", "matplotlib==3.5.1", "mccabe==0.6.1", "message-filters==4.3.2", "micro-ros-msgs==1.0.0", "more-itertools==8.10.0", "mpi4py==3.1.3", "mpmath==0.0.0", "nav-2d-msgs==1.1.6", "nav-msgs==4.2.3", "nav2-common==1.1.6", "nav2-msgs==1.1.6", "nav2-simple-commander==1.0.0", "netifaces==0.11.0", "nose2==0.9.2", "notify2==0.3", "numpy==1.21.5", "oauthlib==3.2.0", "olefile==0.46", "osrf-pycommon==2.0.2", "packaging==21.3", "pbr==5.11.1", "pcl-msgs==1.0.0", "pendulum-msgs==0.20.3", "pexpect==4.8.0", "Pillow==9.0.1", "pip==23.1", "platformdirs==3.1.1", "pluggy==0.13.0", "ply==3.11", "psutil==5.9.0", "ptyprocess==0.7.0", "py==1.10.0", "pyasn1==0.4.8", "pyasn1-modules==0.2.1", "pycairo==1.20.1", "pycodestyle==2.8.0", "pydocstyle==6.1.1", "pydot==1.4.2", "pyflakes==2.4.0", "Pygments==2.11.2", "PyGObject==3.42.1", "PyHamcrest==2.0.2", "PyJWT==2.3.0", "pyOpenSSL==21.0.0", "pyparsing==2.4.7", "PyQt5==5.15.6", "PyQt5-sip==12.9.1", "pyrsistent==0.18.1", "pyserial==3.5", "pytest==6.2.5", "pytest-cov==3.0.0", "python-apt==2.4.0+ubuntu1", "python-dateutil==2.8.1", "python-debian===0.1.43ubuntu1", "python-magic==0.4.24", "python-qt-binding==1.1.1", "pythran==0.10.0", "pytz==2022.1", "PyYAML==6.0", "qt-dotgraph==2.2.2", "qt-gui==2.2.2", "qt-gui-cpp==2.2.2", "qt-gui-py-common==2.2.2", "quality-of-service-demo-py==0.20.3", "rcl-interfaces==1.2.1", "rclpy==3.3.7", "rcutils==5.1.2", "requests==2.28.2", "resource-retriever==3.1.1", "rmw-dds-common==1.6.0", "robot-localization==3.3.1", "roman==3.3", "ros-gz-interfaces==0.244.9", "ros-ign-interfaces==0.244.9", "ros2action==0.18.5", "ros2bag==0.15.4", "ros2cli==0.18.5", "ros2component==0.18.5", "ros2doctor==0.18.5", "ros2interface==0.18.5", "ros2launch==0.19.4", "ros2lifecycle==0.18.5", "ros2multicast==0.18.5", "ros2node==0.18.5", "ros2param==0.18.5", "ros2pkg==0.18.5", "ros2run==0.18.5", "ros2service==0.18.5", "ros2topic==0.18.5", "rosbag2-interfaces==0.15.4", "rosbag2-py==0.15.4", "rosdep==0.22.1", "rosdep-modules==0.22.2", "rosdistro==0.9.0", "rosdistro-modules==0.9.0", "rosgraph-msgs==1.2.1", "rosidl-adapter==3.1.4", "rosidl-cli==3.1.4", "rosidl-cmake==3.1.4", "rosidl-generator-c==3.1.4", "rosidl-generator-cpp==3.1.4", "rosidl-generator-py==0.14.4", "rosidl-parser==3.1.4", "rosidl-runtime-py==0.9.3", "rosidl-typesupport-c==2.0.0", "rosidl-typesupport-cpp==2.0.0", "rosidl-typesupport-fastrtps-c==2.2.0", "rosidl-typesupport-fastrtps-cpp==2.2.0", "rosidl-typesupport-introspection-c==3.1.4", "rosidl-typesupport-introspection-cpp==3.1.4", "rosinstall==0.7.8", "rospkg==1.4.0", "rospkg-modules==1.5.0", "rpyutils==0.2.1", "rqt-action==2.0.1", "rqt-bag==1.1.4", "rqt-bag-plugins==1.1.4", "rqt-console==2.0.2", "rqt-graph==1.3.0", "rqt-gui==1.1.4", "rqt-gui-py==1.1.4", "rqt-msg==1.2.0", "rqt-plot==1.1.2", "rqt-publisher==1.5.0", "rqt-py-common==1.1.4", "rqt-py-console==1.0.2", "rqt-reconfigure==1.1.1", "rqt-service-caller==1.0.5", "rqt-shell==1.0.2", "rqt-srv==1.0.3", "rqt-topic==1.5.0", "scipy==1.8.0", "SecretStorage==3.3.1", "sensor-msgs==4.2.3", "sensor-msgs-py==4.2.3", "service-identity==18.1.0", "setuptools==59.6.0", "shape-msgs==4.2.3", "six==1.16.0", "slam-toolbox==2.6.4", "smclib==3.0.2", "snowballstemmer==2.2.0", "sos==4.4", "soupsieve==2.3.1", "speedtest-cli==2.1.3", "sros2==0.10.4", "ssh-import-id==5.11", "statistics-msgs==1.2.1", "std-msgs==4.2.3", "std-srvs==4.2.3", "stereo-msgs==4.2.3", "stevedore==5.0.0", "sympy==1.9", "systemd-python==234", "teleop-twist-keyboard==2.3.2", "tf2-geometry-msgs==0.25.2", "tf2-kdl==0.25.2", "tf2-msgs==0.25.2", "tf2-py==0.25.2", "tf2-ros-py==0.25.2", "tf2-tools==0.25.2", "theora-image-transport==2.5.0", "toml==0.10.2", "topic-monitor==0.20.3", "trajectory-msgs==4.2.3", "turtlesim==1.4.2", "Twisted==22.1.0", "ubuntu-advantage-tools==8001", "ufoLib2==0.13.1", "ufw==0.36.1", "unattended-upgrades==0.1", "unicodedata2==14.0.0", "unique-identifier-msgs==2.2.1", "urllib3==1.26.5", "vcstool==0.3.0", "vcstools==0.1.42", "virtualenv==20.20.0", "virtualenv-clone==0.5.7", "virtualenvwrapper==4.8.4", "vision-msgs==4.1.0", "visualization-msgs==4.2.3", "wadllib==1.3.6", "webencodings==0.5.1", "wheel==0.37.1", "wstool==0.1.17", "xacro==2.0.8", "zipp==1.0.0", "zope.interface==5.4.0" ], "usb": [ "NoLib" ], "uname": [ "Linux bubble 5.15.0-1026-raspi #28-Ubuntu SMP PREEMPT Fri Mar 10 14:28:52 UTC 2023 aarch64 aarch64" ] } ```
Packages ``` Package: ros-humble-depthai Version: 2.20.2-1jammy.20230202.204256 Priority: optional Section: misc Maintainer: Sachin Guruswamy Installed-Size: 20.1 MB Depends: libc6 (>= 2.34), libgcc-s1 (>= 4.5), libopencv-core4.5d (>= 4.5.4+dfsg), libopencv-imgproc4.5d (>= 4.5.4+dfsg), libstdc++6 (>= 12), libopencv-dev, libusb-1.0-0-dev, nlohmann-json3-dev, ros-humble-ros-workspace Homepage: https://www.luxonis.com/ Download-Size: 10.3 MB APT-Manual-Installed: no APT-Sources: http://packages.ros.org/ros2/ubuntu jammy/main arm64 Packages Description: DepthAI core is a C++ library which comes with firmware and an API to interact with OAK Platform Package: ros-humble-depthai-ros Version: 2.6.4-1jammy.20230303.010530 Priority: optional Section: misc Maintainer: sachin Installed-Size: 42.0 kB Depends: ros-humble-depthai, ros-humble-depthai-bridge, ros-humble-depthai-examples, ros-humble-depthai-ros-driver, ros-humble-depthai-ros-msgs, ros-humble-ros-workspace Download-Size: 5,232 B APT-Sources: http://packages.ros.org/ros2/ubuntu jammy/main arm64 Packages Description: The depthai-ros package Package: ros-humble-depthai-bridge Version: 2.6.4-1jammy.20230302.223904 Priority: optional Section: misc Maintainer: Sachin Guruswamy Installed-Size: 69.4 MB Depends: libc6 (>= 2.32), libgcc-s1 (>= 3.3.1), libopencv-core4.5d (>= 4.5.4+dfsg), libopencv-imgcodecs4.5d (>= 4.5.4+dfsg), libopencv-imgproc4.5d (>= 4.5.4+dfsg), libstdc++6 (>= 11), libboost-dev, libopencv-dev, ros-humble-camera-info-manager, ros-humble-cv-bridge, ros-humble-depthai, ros-humble-depthai-ros-msgs, ros-humble-image-transport, ros-humble-rclcpp, ros-humble-robot-state-publisher, ros-humble-ros-environment, ros-humble-sensor-msgs, ros-humble-std-msgs, ros-humble-stereo-msgs, ros-humble-vision-msgs, ros-humble-xacro, ros-humble-ros-workspace Download-Size: 13.4 MB APT-Sources: http://packages.ros.org/ros2/ubuntu jammy/main arm64 Packages Description: The depthai_bridge package Package: ros-humble-depthai-ros-msgs Version: 2.6.4-1jammy.20230302.210403 Priority: optional Section: misc Maintainer: Sachin Guruswamy Installed-Size: 1,310 kB Depends: libc6 (>= 2.17), libgcc-s1 (>= 3.3.1), libpython3.10 (>= 3.10.0), libstdc++6 (>= 11), ros-humble-fastcdr, ros-humble-builtin-interfaces, ros-humble-geometry-msgs, ros-humble-rclcpp, ros-humble-rosidl-default-generators, ros-humble-sensor-msgs, ros-humble-std-msgs, ros-humble-vision-msgs, ros-humble-ros-workspace Download-Size: 106 kB APT-Sources: http://packages.ros.org/ros2/ubuntu jammy/main arm64 Packages Description: Package to keep interface independent of the driver Package: ros-humble-depthai-ros-driver Version: 2.6.4-1jammy.20230303.005120 Priority: optional Section: misc Maintainer: Adam Serafin Installed-Size: 844 kB Depends: libc6 (>= 2.34), libconsole-bridge1.0 (>= 1.0.1+dfsg2), libgcc-s1 (>= 3.3.1), libopencv-core4.5d (>= 4.5.4+dfsg), libopencv-imgproc4.5d (>= 4.5.4+dfsg), libopencv-stitching4.5d (>= 4.5.4+dfsg), libstdc++6 (>= 11), ros-humble-ament-cmake-auto, ros-humble-camera-calibration, ros-humble-cv-bridge, ros-humble-depthai, ros-humble-depthai-bridge, ros-humble-diagnostic-msgs, ros-humble-image-pipeline, ros-humble-image-transport, ros-humble-image-transport-plugins, ros-humble-rclcpp, ros-humble-rclcpp-components, ros-humble-sensor-msgs, ros-humble-std-msgs, ros-humble-std-srvs, ros-humble-vision-msgs, ros-humble-ros-workspace Download-Size: 213 kB APT-Sources: http://packages.ros.org/ros2/ubuntu jammy/main arm64 Packages Description: Depthai ROS Monolithic node. ```
everskies commented 1 year ago

Update:

Turns out that the data is being published, but only once every 10 or so launches of camera.launch.py.

I'm currently checking if this is a hardware issue (either power related, or something else)

Edit: Does not appear to be the case unfortunately, I've tried some examples from depthai-python with the IR flood and project set to its maximum value, and ran a model at the same time, everything was stable.

HanDaSeul commented 1 year ago

Did you check the issue? There's nothing problem with ROS2 foxy but problem with ROS noetic. It doesn't publish imu at all when I run camera.launch. But stereo_inertial.launch in the dai_examples publish imu.

Serafadam commented 1 year ago

Hi @everskies, from your debug information it seems that your ROS packages are a little behind the official ones, could you upgrade and check again? Also, which camera model is that? @HanDaSeul same here, could you post your camera model and packages version?

HanDaSeul commented 1 year ago

@Serafadam Replied here. https://github.com/luxonis/depthai-ros/issues/304#issuecomment-1533244754