`setFeature` function doesn't work on RealSenseD455

[andrearosasco]

I wrote a short Python script to connect to a realsense device and adjust different parameters. The script works on iCub2.7 but not on ergoCub and R1. I think this might be somehow connected to the camera model as both R1 and ergoCub use a D455 while iCub2.7 has a D415.

In every test the device has been started using the yarprobotinterface and not yarpdev. This is a small script to test the correct behavior:

import yarp
import time
from yarp import IFrameGrabberControls

props = yarp.Property()
props.put('device', 'RGBDSensorClient')
props.put('localImagePort', '/RSGui/rgbImage:i')
props.put('localDepthPort', '/RSGui/depthImage:i')
props.put('localRpcPort', '/RSGui/rpc:i')

props.put('remoteImagePort', '/depthCamera/rgbImage:o')
props.put('remoteDepthPort', '/depthCamera/depthImage:o')
props.put('remoteRpcPort', '/depthCamera/rpc:i')

driver = yarp.PolyDriver(props)

yarp.Network.disconnect('/depthCamera/rgbImage:o', '/RSGui/rgbImage:i')
yarp.Network.disconnect('/depthCamera/depthImage:o', '/RSGui/depthImage:i')

iface = driver.viewIFrameGrabberControls()

for _ in range(10):
    iface.setFeature(yarp.YARP_FEATURE_GAIN, 1.0)
    time.sleep(1)
    iface.setFeature(yarp.YARP_FEATURE_GAIN, 0.0)
    time.sleep(1)

As a quick workaround, I'm working on setting the camera parameter directly, using the pyrealsense2 package. Apparently, even with the yarp device connected to the camera, it is possible to connect and modify the parameters.

[andrearosasco]

I have tried connecting to a D455 and controlling the gain directly through librealsense and everything was working as expected. This is the script I used:

import time
import pyrealsense2 as rs

print('Connecting...')
pipeline = rs.pipeline()
config = rs.config()
config.enable_stream(rs.stream.depth)
config.enable_stream(rs.stream.color)
profile = pipeline.start(config)

rgb_sensor = profile.get_device().query_sensors()[1]
print('Connected')

while True:
    rgb_sensor.set_option(rs.option.gain, 0)
    time.sleep(1)
    rgb_sensor.set_option(rs.option.gain, 127)
    time.sleep(1)

The problem is that if the yarp realsense device is already using the realsense the script above can't connect to it and to change the parameters. This is weird since, if another process is reading images from the realsense using librealsense, the script can still connect to it and change gain, exposure, etc.

[Nicogene]

Hi @andrearosasco, not sure if you already found it but here is how is implemented in the device

https://github.com/robotology/yarp-device-realsense2/blob/d4da73fa2336c40d7d479bb8ee87c317f2128c9b/src/devices/realsense2/realsense2Driver.cpp#L1235-L1243

where setOption

https://github.com/robotology/yarp-device-realsense2/blob/d4da73fa2336c40d7d479bb8ee87c317f2128c9b/src/devices/realsense2/realsense2Driver.cpp#L233-L261

It seems that the calls are the same of the one of the librealsense example

[andrearosasco]

Thanks, @Nicogene! Yes, everything looks the same. The only thing I could think so is that for some reason on the d455 sensor->supports(option) returns false even if the option is supported.

[Nicogene]

Thanks, @Nicogene! Yes, everything looks the same. The only thing I could think so is that for some reason on the d455 sensor->supports(option) returns false even if the option is supported.

Good catch! Maybe it is a bug that can be reported in librealsense repository.

[xEnVrE]

Today I checked this issue and I found that - on ergoCub at least - when I try to set the options via RPC the device outputs an error due to this exception:

https://github.com/robotology/yarp-device-realsense2/blob/d4da73fa2336c40d7d479bb8ee87c317f2128c9b/src/devices/realsense2/realsense2Driver.cpp#L191-L203

where e.what() contains failed to set power state, which seems to me quite unrelated.

cc @Nicogene

[andrearosasco]

So I just tested the code again and it looks like it works. I didn't change anything but the version of yarp and of the yarp-device-realsense2 on ergoCub have probably changed at some point .

Current versions Yarp: 3.8.1+157-20231016.1+gita0d408b7c yapr-device-realsense: 47430cb4daa41aa42e5f6dc0ddb9d3ad553f6e7e librealsense: b59b13671658910fc453a4a6bbd61f13ba6e83cc

[traversaro]

Another thing that may play a role here is the firmware version of the realsense. On ergoCubSN001 we could not get yarp-device-realsense2 to work until we updated the firmware version via the integrated gui of librealsense.

[xEnVrE]

Just to clarify, this has been tested on the new ergoCubSN001 while the issue was raised on the previous ergoCubSN000.

cc @Nicogene

[traversaro]

Just to clarify, this has been tested on the new ergoCubSN001 while the issue was raised on the previous ergoCubSN000.

cc @Nicogene

Indeed, I guess that the firmware version between the two realsense may be different.