yarp-device-realsense2

This is the realsense device for YARP. It supports the Intel® RealSense cameras.

The Intel® RealSense™ cameras currently compatible with YARP are:

• Depth Camera D405
• Depth Camera D415
• Depth Camera D435
• Depth Camera D455
• LiDAR Camera L515
• Depth Camera with IMU D435i
• Tracking Camera T256

License

This software may be modified and distributed under the terms of the BSD-3-Clause license. See the accompanying LICENSE file for details.

The realsense2 device uses the librealsense library, released under the Apache-2.0 license. See the relative documentation for the terms of the license.

How to use Intel RealSense cameras as a YARP device

Dependencies

Before proceeding further, please install the following dependencies:

• librealsense

Configure YARP

Before proceeding further, please enable the following CMake flags in YARP:

ENABLE_yarpmod_RGBDSensorWrapper
ENABLE_yarpmod_RGBDSensorClient
ENABLE_yarpcar_mjpeg
ENABLE_yarppm_depthimage_to_rgb

:bulb: NOTE: If you use YARP < 3.5, instead of the option ENABLE_yarppm_depthimage_to_rgb you should enable the option ENABLE_yarpcar_depthimage2.

Build and install yarp-device-realsense2

mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=<installation_path> ..
make
make install

In order to make the device detectable, add <installation_path>/share/yarp to the YARP_DATA_DIRS environment variable of the system.

Alternatively, if YARP has been installed using the robotology-superbuild, it is possible to use <directory-where-you-downloaded-robotology-superbuild>/build/install as the <installation_path>.

How to use a RealSense D435

As yarpmanager application

You can write a yarp application to launch the camera and see the RGB and Depth images. Here is a simple example:

<application>
    <name>simpleRealsense</name>
    <description>Description</description>
    <version>1.0</version>

    <module>
      <name>yarpdev</name>
          <parameters> --from sensors/RealSense_conf.ini </parameters>
      <node>localhost</node>
    </module>  

    <module>
      <name>yarpview</name>
      <parameters>--name /view/rgb:i</parameters>
      <node>localhost</node>
    </module>

    <module>
      <name>yarpview</name>
      <parameters>--name /view/depth:i</parameters>
      <node>localhost</node>
    </module>

    <connection>
      <from>/depthCamera/rgbImage:o</from>
      <to> /view/rgb:i</to>
      <protocol>mjpeg</protocol>
    </connection>

    <connection>
      <from>/depthCamera/depthImage:o</from>
      <to>/view/depth:i</to>
      <protocol>udp+recv.portmonitor+type.dll+file.depthimage_to_rgb</protocol>
    </connection>

</application>

:bulb: NOTE: When using the camera locally and, not over a network, the user can substitute the protocol used for visualizing the depth image as follows:

<protocol>unix_stream+recv.portmonitor+type.dll+file.depthimage_to_rgb</protocol>

:bulb: NOTE: When using YARP < 3.5, the final part of the protocol string should be changed from +file.depthimage_to_rgb to +file.depthimage2.

From the command line

Set the YARP_ROBOT_NAME to either CER01, CER02 or CER03. Then, run the command:

yarpdev --from sensors/RealSense_conf.ini

By setting the YARP_ROBOT_NAME, YARP finds the configuration file RealSense_conf.ini automatically.

RealSense_conf.ini contains the following settings:

device       RGBDSensorWrapper
subdevice    realsense2
name         /depthCamera

[SETTINGS]
depthResolution (480 270)    #Other possible values (424 240) or (640 480)
rgbResolution   (424 240)    #Other possible values (424 240) or (640 480)
framerate       30
enableEmitter   true
needAlignment   true
alignmentFrame  RGB

[HW_DESCRIPTION]
clipPlanes (0.2 10.0)

:warning: WARNING: the user might have to change the parameters depthResolution and rgbResolution. These parameters change depending on the firmware version installed on the used camera.

How to use a RealSense D405

To use this model of camera you can follow the same instructions above with the following notable differences:

• this camera does not have an active emitter system, hence it should be disabled as not supported;
• this camera does not have separate RGB and depth sensors - it only has a depth sensor that also provides RGB images - hence only the depth resolution should be set;
• this camera does not need active software-based alignment per construction - see here - hence the alignment should be disabled;
• this camera is a short-range camera, hence it is suggested to provide an adequately low lower bound for the clipping planes.

Considering the above, a possible configuration file might be the following:

device       RGBDSensorWrapper
subdevice    realsense2
name         /depthCamera

[SETTINGS]
depthResolution (640 480) # <-- Same resolution as the rgbResolution
framerate       30
alignmentFrame  None      # <-- alignment set to None
                          # <-- 'enableEmitter' removed as not supported

[HW_DESCRIPTION]
clipPlanes (0.01 10.0)    # <-- lower bound for clipping planes set to 0.01 meters

How to use a RealSense D435i

:construction: UNDER CONSTRUCTION :construction:

How to use a RealSense T256

From the command line

The device retrieving the RealSense T256 data is the multipleanalogsensorserver and it must be run as follows:

yarpdev --device multipleanalogsensorsserver --name /t256 --period 10 --subdevice realsense2Tracking

:bulb: NOTE: the user should specify the parameters --name and --period as needed.

The data from a RealSense T256 will be streamed on the port named /t256/measures:o following the format described in the SensorStreamingData class.

The type on information currently made available are:

• the Gyroscope measures,
• the Accelerometer measures, and
• the Pose (position and orientation).

Device documentation

This device driver exposes the yarp::dev::IRGBDSensor and yarp::dev::IFrameGrabberControls interfaces to read the images and operate on the available settings. See the documentation for more details about each interface.

This device is paired with its server called RGBDSensorWrapper to stream the images and perform remote operations.

The configuration file is subdivided into 2 major sections called SETTINGS and HW_DESCRIPTION.

The SETTINGS section is meant for read/write parameters, meaning parameters which can be get and set by the device. A common case of setting is the image resolution in pixel. This setting will be read by the device and it'll be applied in the startup phase. If the setting fails, the device will terminate the execution with a error message.

The HW_DESCRIPTION section is meant for read only parameters which describe the hardware property of the device and cannot be provided by the device through software API. A common case is the 'Field Of View' property, which may or may not be supported by the physical device. When a property is present in the HW_DESCRIPTION group, the YARP RGBDSensorClient will report this value when asked for and setting will be disabled. This group can also be used to by-pass realsense2 API in case some functionality is not correctly working with the current device. For example the clipPlanes property may return incorrect values or values using non-standard measurement unit. In this case using the HW_DESCRIPTION, a user can override the value got from OpenNI2 API with a custom value.

Note: Parameters inside the HW_DESCRIPTION are read only, so the relative set functionality will be disabled.

Note: For parameters which are neither in SETTINGS nor in HW_DESCRIPTION groups, read / write functionality is assumed but not initial setting will be performed. Device will start with manufacturer default values.

Warning: A single parameter cannot be present into both SETTINGS and HW_DESCRIPTION groups.

Warning whenever more then one value is required by the setting, the values must be in parentheses!

Parameters used by this device are:

Configuration file using .ini format, for using as RGBD device:

device       RGBDSensorWrapper
subdevice    realsense2
name         /depthCamera

[SETTINGS]
depthResolution (640 480)    #Note the parentheses
rgbResolution   (640 480)
framerate       30
enableEmitter   true
alignmentFrame  RGB

[HW_DESCRIPTION]
clipPlanes (0.2 10.0)

Configuration file using .ini format, for using as stereo camera:

device       grabberDual
subdevice    realsense2
name         /stereoCamera
capabilities RAW
stereoMode   true

[SETTINGS]
rgbResolution   (640 480)
depthResolution   (640 480)
framerate       30
enableEmitter   false

[HW_DESCRIPTION]
clipPlanes (0.2 10.0)

Maintainers

This repository is maintained by:

	@traversaro