search

pollen-robotics / depthai-ros

Fork of depthai-ros, provides ROS bindings around Luxonis' C++ library
MIT License
1 stars 0 forks source link

readme

README Pollen :)

Yo, Fork the Luxonis, avec ce fix: https://github.com/luxonis/depthai-ros/issues/248 Et ce fix: https://github.com/luxonis/depthai-ros/issues/258

Ce que je lance :

ros2 launch depthai_ros_driver camera_yolo.launch.py

Il faut installer une lib Pyhton :

pip3 install empy

/!\ Il y a des chemin en dur à update dans depthai_ros_driver/config/camera_yolo.yaml:

i_model_path: ...
i_nn_config_path: ...

Le blob se trouve ici:

https://github.com/pollen-robotics/simple_grasp_pipe/blob/master/simple_grasp_pipe/utils/yolo-v7-tiny_openvino_2021.4_6shave.blob

Bisous

Depthai ROS Repository

Hi and welcome to the main depthai-ros respository! Here you can find ROS related code for OAK cameras from Luxonis. Don't have one? You can get them here!

Main features:

You can develop your ROS applications in following ways:

Supported ROS versions:

For usage check out respective git branches.

Install from ros binaries

Add USB rules to your system

echo 'SUBSYSTEM=="usb", ATTRS{idVendor}=="03e7", MODE="0666"' | sudo tee /etc/udev/rules.d/80-movidius.rules
sudo udevadm control --reload-rules && sudo udevadm trigger

Install depthai-ros. (Available for Noetic, foxy, galactic and humble) sudo apt install ros-<distro>-depthai-ros

Install from source

Install dependencies

The following script will install depthai-core and update usb rules and install depthai devices

sudo wget -qO- https://raw.githubusercontent.com/luxonis/depthai-ros/main/install_dependencies.sh | sudo bash

if you don't have opencv installed then try sudo apt install libopencv-dev

if you don't have rosdep installed and not initialized please execute the following steps:

  1. sudo apt install python-rosdep(melodic) or sudo apt install python3-rosdep
  2. sudo rosdep init
  3. rosdep update

Setting up procedure

The following setup procedure assumes you have cmake version >= 3.10.2 and OpenCV version >= 4.0.0. We selected dai_ws as the name for a new folder, as it will be our depthai ros workspace.

  1. mkdir -p dai_ws/src
  2. cd dai_ws/src
  3. git clone --branch <ros-distro> https://github.com/luxonis/depthai-ros.git
  4. cd ..
  5. rosdep install --from-paths src --ignore-src -r -y
  6. source /opt/ros/<ros-distro>/setup.bash
  7. catkin_make_isolated (For ROS1) MAKEFLAGS="-j1 -l1" colcon build (for ROS2)
  8. source devel/setup.bash (For ROS1) & source install/setup.bash (for ROS2)

Note If you are using a lower end PC or RPi, standard building may take a lot of RAM and clog your PC. To avoid that, you can use build.sh command from your workspace (it just wraps colcon commands): ./src/depthai-ros/build.sh

Docker

You can additionally build and run docker images on your local machine. To do that, add USB rules as in above step.

Running prebuilt images

Each tagged version has it's own prebuilt docker image. To download and run it:

xhost +local:docker

to enable GUI tools such as rviz or rqt.

Then

docker run -it -v /dev/:/dev/  --privileged -e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix luxonis/depthai-ros:noetic-latest [CMD]

Where [CMD] is what's going to be executed when container is initially run and could be for example:

Building

Clone the repository and inside it run (it matters on which branch you are on):

docker build --build-arg USE_RVIZ=1 -t depthai-ros .

If you find out that you run out of RAM during building, you can also set BUILD_SEQUENTIAL=1 to build packages one at a time, it should take longer, but use less RAM.

RUN_RVIZ arg means rviz will be installed inside docker. If you want to run it you need to also execute following command (you'll have to do it again after restarting your PC):

xhost +local:docker

Then you can run your image in following way:

docker run -it -v /dev/:/dev/ --privileged -e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix depthai-ros

will run an interactive docker session. You can also try:

docker run -it -v /dev/:/dev/ --privileged -e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix depthai-ros roslaunch depthai_examples stereo_inertial_node.launch.py

to run a launch file of your choice.

NOTE ROS2 Humble docker image uses Cyclone as RMW implementation.

Running docker image on ROS1

docker run -it -v /dev/:/dev/ --privileged -e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix depthai_ros roslaunch depthai_examples stereo_inertial_node.launch

Will only start stereo_inertial_node launch file (you can try different commands).

Running docker image on ROS2

docker run -it -v /dev/:/dev/ --privileged -e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix depthai_ros roslaunch depthai_examples stereo_inertial_node.launch.py

Depthai ROS Driver

Currently, recommended way to launch cameras is to use executables from depthai_ros_driver package.

This runs your camera as a ROS2 Component and gives you the ability to customize your camera using ROS parameters. Paramerers that begin with r_ can be freely modified during runtime, for example with rqt. Parameters that begin with i_ are set when camera is initializing, to change them you have to call stop and start services. This can be used to hot swap NNs during runtime, changing resolutions, etc. Below you can see some examples:

Setting RGB parameters

By default RGB camera outputs ISP frame. To set custom width and height of output image, you can set i_isp_num and i_isp_den which scale image dimensions (2 and 3 by default, so from 1920x1080 to 1280x720), note for RGBD alignment to work resulting width and height must be divisible by 16.

Additionally you can set i.output_isp: false to use video output and set custom size using i_width and i_height parameters.

Setting Stereo parameters

Setting IMU parameters

Parameters:

Stopping/starting camera for power saving/reconfiguration

Stopping camera also can be used for power saving, as pipeline is removed from the device. Topics are also removed when camera is stopped.

As for the parameters themselves, there are a few crucial ones that decide on how camera behaves.

To do that, you can create a custom package (let's say test_plugins), create an executable in that package (test_plugins.cpp). Inside that file, define a cusom plugin that inherits from depthai_ros_driver::pipeline_gen::BasePipeline and overrides createPipeline method.

After that export plugin, for example:

#include <pluginlib/class_list_macros.hpp>
PLUGINLIB_EXPORT_CLASS(test_plugins::Test, depthai_ros_driver::pipeline_gen::BasePipeline)

Add plugin definition:

<library path="test_plugins">
    <class type="test_plugins::Test" base_class_type="depthai_ros_driver::pipeline_gen::BasePipeline">
        <description>Test Pipeline.</description>
    </class>
</library>

Now you can use created plugin as pipeline, just set camera.i_pipeline_type to test_plugins::Test.

To use provided example NN's, you can set the path to:

All available camera-specific parameters and their default values can be seen in depthai_ros_driver/config/camera.yaml.

Currently, we provide few examples:

Specific camera configurations:

PoE Cameras

Since PoE cameras use protocol that has lower throughput than USB, running default camera launch can result in lags depending on chosen resolution/fps. To combat this issue, you can use encoded frames, which let you keep desired resolution/fps at the cost of image quality reduction due to compression. One additional difference is that subpixel depth filtering is disabled in this mode. To enable low_bandwidth, for example for rgb camera, change parameters:

Recalibration

If you want to use other calibration values than the ones provided by the device, you can do it in following ways:

Depthai filters

depthai_filters contains small composable node examples that show how to work with data from multiple topics. Available filters:

Using external sources for NN inference or Stereo Depth

There is a possibility of using external image topics as input for NNs or Depth calculation.

Example scenarios:

  1. We want to get segmentation or 2D detection output based on images published by usb camera node. This can be achieved by setting rgb.i_simulate_from_topic parameter to true. This creates sensor_msgs/Image subscriber listening by default on /<node_name>/rgb/input topic that passes data into the pipeline on each callback. Topic names can be changed either by classic ROS topic remapping or setting rgb.i_simulated_topic_name to a desired name. By default, original sensor node still runs and publishes data. Setting rgb.i_disable_node to true will prevent it from spawning. Check det2d_usb_cam_overlay.launch.py in `depthai_filters to see it in action.

  2. Calculating depth - both left and right sensor nodes can be setup as in the example above to calculate calculate depth/disparity from external topics. Note that for this to work properly you need specify:

    • left.i_board_socket_id: 1
    • right.i_board_socket_id: 2
    • Default stereo input size is set to 1280x720, in case of different image size, To set custom ones, set stereo.i_set_input_size: true and adjust stereo.i_input_width and stereo.i_input_height accordingly.
    • external calibration file path using camera.i_external_calibration_path parameter. To get calibration from the device you can either set camera.i_calibration_dump to true or call save_calibration service. Calibration will be saved to /tmp/<mx_id>_calibration.json. An example can be seen in stereo_from_rosbag.launch.py in depthai_ros_driver

Executing an example

ROS1

  1. cd dai_ws (Our workspace)
  2. source devel/setup.bash
  3. roslaunch depthai_examples stereo_inertial_node.launch - example node For more examples please check the launch files.

ROS2

  1. cd dai_ws (Our workspace)
  2. source install/setup.bash
  3. ros2 launch depthai_examples stereo_inertial_node.launch.py - example node For more examples please check the launch files.

Running Examples

Mobilenet Publisher:

ROS1:

OAK-D
roslaunch depthai_examples mobile_publisher.launch camera_model:=OAK-D
OAK-D-LITE
roslaunch depthai_examples mobile_publisher.launch camera_model:=OAK-D-LITE
With visualizer
roslaunch depthai_examples mobile_publisher.launch | rqt_image_view -t /mobilenet_publisher/color/image

ROS2:

OAK-D
ros2 launch depthai_examples mobile_publisher.launch.py camera_model:=OAK-D
OAK-D-LITE
ros2 launch depthai_examples mobile_publisher.launch.py camera_model:=OAK-D-LITE

Testing results

Users can write Custom converters and plug them in for bridge Publisher.

If there a standard Message or usecase for which we have not provided a ros msg or converter feel free to create a issue or reach out to us on our discord community. We would be happy to add more.

Developers guide

For easier development inside isolated workspace, one can use Visual Studio Code with DevContainers plugin, to do that: