search

orbbec / OrbbecSDK_ROS2

OrbbecSDK ROS2 wrapper
https://orbbec.github.io/OrbbecSDK_ROS2/
Apache License 2.0
57 stars 19 forks source link
3d camera depth-camera robotics ros2

readme

Orbbec ROS2 SDK

stable version

Orbbec SDK ROS 2 is a wrapper for the Orbbec 3D camera that provides seamless integration with the ROS 2 environment. It supports ROS 2 Foxy, Humble, and Jazzy distributions.

Table of Contents

Installation Instructions

Install ROS 2

If your ROS 2 command does not auto-complete, put the following two lines into your .bashrc or .zshrc

eval "$(register-python-argcomplete3 ros2)"
eval "$(register-python-argcomplete3 colcon)"

Create colcon workspace

mkdir -p ~/ros2_ws/src

Get source code

cd ~/ros2_ws/src
git clone https://github.com/orbbec/OrbbecSDK_ROS2.git

Install deb dependencies

# assume you have sourced ROS environment, same blow
sudo apt install libgflags-dev nlohmann-json3-dev  \
ros-$ROS_DISTRO-image-transport ros-$ROS_DISTRO-image-publisher ros-$ROS_DISTRO-camera-info-manager \
ros-$ROS_DISTRO-diagnostic-updater ros-$ROS_DISTRO-diagnostic-msgs

Install udev rules.

cd  ~/ros2_ws/src/OrbbecSDK_ROS2/orbbec_camera/scripts
sudo bash install_udev_rules.sh
sudo udevadm control --reload-rules && sudo udevadm trigger

Getting start

cd ~/ros2_ws/
# build release, Default is Debug
colcon build --event-handlers  console_direct+  --cmake-args  -DCMAKE_BUILD_TYPE=Release

Launch camera node

. ./install/setup.bash
ros2 launch orbbec_camera astra.launch.py # or other launch file, see below table
. ./install/setup.bash
rviz2

Select the topic you want to display

ros2 topic list
ros2 service list
ros2 param list
ros2 service call /camera/get_device_info orbbec_camera_msgs/srv/GetDeviceInfo '{}'
ros2 service call /camera/get_sdk_version orbbec_camera_msgs/srv/GetString '{}'

ros2 service call /camera/get_color_exposure orbbec_camera_msgs/srv/GetInt32 '{}'

If your check ir or depth, please change /camera/get_color_exposure to /camera/get_ir_exposure or /camera/get_depth_exposure, Same below.

ros2 service call /camera/get_color_gain orbbec_camera_msgs/srv/GetInt32 '{}'
ros2 service call /camera/get_white_balance orbbec_camera_msgs/srv/GetInt32 '{}'
ros2 service call /camera/set_color_auto_exposure std_srvs/srv/SetBool '{data: false}'
ros2 service call /camera/set_white_balance orbbec_camera_msgs/srv/SetInt32 '{data: 4600}'
ros2 service call  /camera/set_laser_enable std_srvs/srv/SetBool "{data: true}"
ros2 service call /camera/toggle_ir std_srvs/srv/SetBool "{data : true}"

ros2 service call /camera/save_point_cloud std_srvs/srv/Empty "{}"

Launch parameters

The following are the launch parameters available:

IMPORTANT: Please carefully read the instructions regarding software filtering settings at this link. If you are uncertain, do not modify these settings.

Predefined presets

Preset Features Recommended use cases
Default - Best visual perception
- Overall good performance in accuracy, fill rate, tiny objects, etc.		 - Generic
- Robotics
Hand - Clear hand and finger edges - Gesture recognition
High Accuracy - Depth of high confidence
- Barely noise depth values
- Lower fill rate		 - Collision avoidance
- Object scanning
High Density - Higher fill rate
- More tiny objects
- May suffer from noise depth values		 - Object recognition
- Pick & place
- Foreground & background animation
Medium Density - Balanced performance in fill rate and accuracy
- In comparison to Default: lower fill rate, better edge quality		 - Generic and alternative to Default
Custom - User defined Preset
- Derived from Presets above, with customized modifications, e.g. a new configuration for the post-processing pipeline, modified mean intensity set point of depth AE function, etc.		 - Better depth performance achieved using customized configurations in comparison to using predefined presets
- For well-established custom configurations

Choose the appropriate preset name based on your specific use case and set it as the value for the device_preset parameter.

Depth work mode switch

Orbbec SDK ROS 2 supports the depth work mode switch. The depth work mode switch is supported by Gemini 2, Gemini 2 L, and Femto and Femto Bolt cameras.

# Depth work mode support is as follows:
# Unbinned Dense Default
# Unbinned Sparse Default
# Binned Sparse Default
# Obstacle Avoidance
DeclareLaunchArgument('depth_work_mode', default_value='')
ros2 run orbbec_camera list_depth_work_mode_node

Configuration of depth NFOV and WFOV modes

For the Femto Mega and Femto Bolt devices, the NFOV and WFOV modes are implemented by configuring the resolution of Depth and IR in the launch file. In launch file, depth_width、depth_height、ir_width、ir_height represents the resolution of the depth and the resolution of the IR. The frame fps and resolution of IR must be consistent with the depth. The correspondence between different modes and resolutions is as follows:

All available service for camera control

The name of the following service already expresses its function. However, it should be noted that the corresponding set_[ir|depth|color]* and get[ir|depth|color]* services are only available if you set enable[ir|depth|color] to true in the stream that corresponds to the argument of the launch file.

All available topics

Network device enumeration

Currently, the network device enumeration function is supported only by the Femto Mega device. When accessing this device over the network, if enumerate_net_device is set to true, the device will be automatically enumerated, eliminating the need to configure the IP address in advance or set the enable switch to true. The specific configuration methods are as follows:

Multi-Camera

ros2 run orbbec_camera list_devices_node
from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument, IncludeLaunchDescription, GroupAction, ExecuteProcess
from launch.launch_description_sources import PythonLaunchDescriptionSource
from launch_ros.actions import Node
from ament_index_python.packages import get_package_share_directory
import os

def generate_launch_description():
    # Include launch files
    package_dir = get_package_share_directory('orbbec_camera')
    launch_file_dir = os.path.join(package_dir, 'launch')
    launch1_include = IncludeLaunchDescription(
        PythonLaunchDescriptionSource(
            os.path.join(launch_file_dir, 'gemini2L.launch.py')
        ),
        launch_arguments={
            'camera_name': 'camera_01',
            'usb_port': '6-2.4.4.2',  # replace your usb port here
            'device_num': '2'
        }.items()
    )

    launch2_include = IncludeLaunchDescription(
        PythonLaunchDescriptionSource(
            os.path.join(launch_file_dir, 'gemini2L.launch.py')
        ),
        launch_arguments={
            'camera_name': 'camera_02',
            'usb_port': '6-2.4.1',  # replace your usb port here
            'device_num': '2'
        }.items()
    )

    # If you need more cameras, just add more launch_include here, and change the usb_port and device_num

    # Launch description
    ld = LaunchDescription([
        GroupAction([launch1_include]),
        GroupAction([launch2_include]),
    ])

    return ld

ros2 launch orbbec_camera multi_camera.launch.py

Use hardware decoder to decode JPEG

rockchip and Amlogic

Depends on rockchip-mpp-dev and rockchip-rga-dev, not all systems have these two packages, the names may be different, please search by yourself. Open CMakeLists.txt and set USE_RK_HW_DECODER to ON.

Nvidia Jetson

Depends on: jetson_multimedia_api,libyuv. Open CMakeLists.txt and set USE_NV_HW_DECODER to ON.

Check which profiles the camera supports

ros2 run orbbec_camera list_camera_profile_mode_node

Building a Debian Package

Preparing the Environment

Before starting, install the required tools:

sudo apt install debhelper fakeroot python3-bloom

Configuring ROS Dependencies

Add the following YAML file to your system at /etc/ros/rosdep/sources.list.d/00-orbbec.yaml. Make sure to replace focal with the codename of your Ubuntu version and humble with your ROS2 distribution name:

orbbec_camera_msgs:
  ubuntu:
    focal: [ ros-humble-orbbec-camera-msgs ]

Next, create a new file /etc/ros/rosdep/sources.list.d/50-orbbec.list and add this line to specify the path to the YAML file:

yaml file:///etc/ros/rosdep/sources.list.d/00-orbbec.yaml

Update the rosdep database to reflect these changes:

rosdep update

Building the Package

Navigate to your workspace and build the project:

cd ~/ros2_ws/
colcon build --event-handlers console_direct+ --cmake-args -DCMAKE_BUILD_TYPE=Release
. install/setup.bash
cd src/OrbbecSDK_ROS2/
bash .make_deb.sh

Launch files

product serials launch file
astra+ astra_adv.launch.py
astra mini /astra mini pro /astra pro astra.launch.py
astra mini pro s astra.launch.py
astra2 astra2.launch.py
astra stereo s stereo_s_u3.launch.py
astra pro2 astra_pro2.launch.py
dabai dabai.launch.py
dabai d1 dabai_d1.launch.py
dabai dcw dabai_dcw.launch.py
dabai dw dabai_dw.launch.py
dabai pro dabai_pro.launch.py
deeya deeya.launch.py
femto /femto w femto.launch.py
femto mega femto_mega.launch.py
femto bolt femto_bolt.launch.py
gemini gemini.launch.py
gemini gemini.launch.py
gemini2 / dabai DCL gemini2.launch.py
gemini2L gemini2L.launch.py
gemini e gemini_e.launch.py
gemini e lite gemini_e_lite.launch.py
dabai max dabai_max.launch.py
dabai max pro dabai_max_pro.launch.py
gemini uw gemini_uw.launch.py
dabai dcw2 dabai_dcw2.launch.py
dabai dw2 dabai_dw2.launch.py
gemini ew gemini_ew.launch.py
gemini ew lite gemini_ew_lite.launch.py
gemini 330 series gemini_330_series.launch.py

All launch files are essentially similar, with the primary difference being the default values of the parameters set for different models within the same series. Differences in USB standards, such as USB 2.0 versus USB 3.0, may require adjustments to these parameters. If you encounter a startup failure, please carefully review the specification manual. Pay special attention to the resolution settings in the launch file, as well as other parameters, to ensure compatibility and optimal performance.

Product support

Please refer to the OrbbecSDK supported products: Product Support

DDS Tuning

The default DDS settings (Galactic) may not be optimal for data transmission. Different DDS settings can have varying performance. In this example, we use CycloneDDS. For more detailed information, please refer to the ROS DDS Tuning。

● Edit cyclonedds configuration file

sudo gedit /etc/cyclonedds/config.xml

Add

<?xml version="1.0" encoding="UTF-8"?>
<CycloneDDS xmlns="https://cdds.io/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="https://cdds.io/confighttps://raw.githubusercontent.com/eclipse-cyclonedds/cyclonedds/master/etc/cyclonedds.xsd">
    <Domain id="any">
        <General>
            <NetworkInterfaceAddress>lo</NetworkInterfaceAddress>
            <AllowMulticast>false</AllowMulticast>
        </General>
        <Internal>
            <MinimumSocketReceiveBufferSize>16MB</MinimumSocketReceiveBufferSize>
        </Internal>
        <Discovery>
            <ParticipantIndex>auto</ParticipantIndex>
            <MaxAutoParticipantIndex>30</MaxAutoParticipantIndex>
            <Peers>
                <Peer address="localhost"/>
            </Peers>
        </Discovery>
    </Domain>
</CycloneDDS>

● Set the environment variables, add to .zshrc or .bashrc

export ROS_DOMAIN_ID=42 # Numbers from 0 to 232
export ROS_LOCALHOST_ONLY=1
export CYCLONEDDS_URI=file:///etc/cyclonedds/config.xml

Tip：to understand why the maximum ROS_DOMAIN_ID is 232, please visit The ROS DOMAIN ID ● Increase UDP receive buffer size Edit

/etc/sysctl.d/10-cyclone-max.conf

Add

net.core.rmem_max=2147483647
net.core.rmem_default=2147483647

Frequently Asked Questions

No Picture from Multiple Cameras

Why are there so many launch files here

Other useful links

License

Copyright 2024 Orbbec Ltd.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this project except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an " AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Other names and brands may be claimed as the property of others