search

IntelRealSense / realsense-ros

ROS Wrapper for Intel(R) RealSense(TM) Cameras
http://wiki.ros.org/RealSense
Apache License 2.0
2.56k stars 1.75k forks source link

Unable to change exposure in HDR mode: hdr_config::send_sub_preset_to_fw() - hw command failed #2899

Closed Finn2708 closed 1 year ago

Finn2708 commented 1 year ago

Hi,

I'm trying to utilize the HDR mode of my D435i to handle high dynamic range environments. However, I'm unable to modify stereo_module/exposure/1 and stereo_module/exposure/2 through rosrun rqt_reconfigure rqt_reconfigure as described in the readme.

My workflow looks like this:

  1. roslaunch realsense2_camera rs_camera.launch filters:=hdr_merge 
    
... logging to /home/weithoff/.ros/log/d61b4b84-6444-11ee-9a6e-7785b070c095/roslaunch-weithoff-CELSIUS-H7510-47262.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://weithoff-CELSIUS-H7510:33137/

SUMMARY

PARAMETERS

NODES /camera/ realsense2_camera (nodelet/nodelet) realsense2_camera_manager (nodelet/nodelet)

auto-starting new master process[master]: started with pid [47288] ROS_MASTER_URI=http://localhost:11311

setting /run_id to d61b4b84-6444-11ee-9a6e-7785b070c095 process[rosout-1]: started with pid [47314] started core service [/rosout] process[camera/realsense2_camera_manager-2]: started with pid [47321] process[camera/realsense2_camera-3]: started with pid [47322] [ INFO] [1696595731.245765460]: Initializing nodelet with 16 worker threads. [ INFO] [1696595731.330129415]: RealSense ROS v2.3.2 [ INFO] [1696595731.330159565]: Built with LibRealSense v2.50.0 [ INFO] [1696595731.330170606]: Running with LibRealSense v2.50.0 [ INFO] [1696595731.366132146]:
[ INFO] [1696595731.659284090]: Device with serial number 918512072160 was found.

[ INFO] [1696595731.659312056]: Device with physical ID 2-3-3 was found. [ INFO] [1696595731.659327591]: Device with name Intel RealSense D435I was found. [ INFO] [1696595731.659627285]: Device with port number 2-3 was found. [ INFO] [1696595731.659647954]: Device USB type: 3.2

[ INFO] [1696595731.687267531]: JSON file is not provided [ INFO] [1696595731.687275014]: ROS Node Namespace: camera [ INFO] [1696595731.687283022]: Device Name: Intel RealSense D435I [ INFO] [1696595731.687288918]: Device Serial No: 918512072160 [ INFO] [1696595731.687296857]: Device physical port: 2-3-3 [ INFO] [1696595731.687305912]: Device FW version: 05.15.00.02 [ INFO] [1696595731.687317501]: Device Product ID: 0x0B3A [ INFO] [1696595731.687324725]: Enable PointCloud: Off [ INFO] [1696595731.687331450]: Align Depth: Off [ INFO] [1696595731.687337715]: Sync Mode: On [ INFO] [1696595731.687368962]: Device Sensors: [ INFO] [1696595731.813003951]: Stereo Module was found. [ INFO] [1696595731.832562984]: RGB Camera was found. [ INFO] [1696595731.832728034]: Motion Module was found. [ INFO] [1696595731.832761932]: (Confidence, 0) sensor isn't supported by current device! -- Skipping... [ INFO] [1696595731.832787374]: Add Filter: hdr_merge [ INFO] [1696595731.833069657]: Add Filter: sequence_id_filter [ INFO] [1696595731.833328427]: num_filters: 2 [ INFO] [1696595731.833383343]: Reading option: stereo_module/exposure/1 [ INFO] [1696595731.834495246]: Reading option: stereo_module/gain/1 [ INFO] [1696595731.835240594]: Reading option: stereo_module/exposure/2 [ INFO] [1696595731.835845963]: Reading option: stereo_module/gain/2 [ INFO] [1696595731.931729029]: Setting Dynamic reconfig parameters. hwmon command 0x80( 5 0 0 0 ) failed (response -7= HW not ready) hwmon command 0x80( 5 0 0 0 ) failed (response -7= HW not ready) hwmon command 0x80( 5 0 0 0 ) failed (response -7= HW not ready) hwmon command 0x80( 5 0 0 0 ) failed (response -7= HW not ready) [ INFO] [1696595735.471410040]: Done Setting Dynamic reconfig parameters. [ INFO] [1696595735.471806007]: depth stream is enabled - width: 848, height: 480, fps: 30, Format: Z16 [ INFO] [1696595735.472681073]: color stream is enabled - width: 1280, height: 720, fps: 30, Format: RGB8

[ INFO] [1696595735.474979301]: Expected frequency for depth = 30.00000 [ INFO] [1696595735.501077028]: Expected frequency for color = 30.00000

06/10 14:35:35,742 WARNING [140330003330816] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0 [ INFO] [1696595735.792643213]: SELECTED BASE:Depth, 0 [ INFO] [1696595735.800362242]: RealSense Node Is Up! [ WARN] [1696595735.829657939]: 06/10 14:35:35,830 WARNING [140330003330816] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0 06/10 14:35:35,880 WARNING [140330003330816] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0 06/10 14:35:35,930 WARNING [140330003330816] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0 06/10 14:35:36,162 WARNING [140330003330816] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0 06/10 14:35:36,212 WARNING [140330003330816] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0 06/10 14:35:36,263 WARNING [140330003330816] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0


2. `rosrun rqt_reconfigure rqt_reconfigure`
3. Set `sequence_id` from `UVC (0)` to `1 (1)` (sequence_name: 0, sequence_size: 2):

[ INFO] [1696595778.899847823]: call to set stereo_module/set_parameters/gain to 16 succeeded [ INFO] [1696595778.900906304]: call to set stereo_module/set_parameters/exposure to 7500 succeeded 06/10 14:36:18,899 WARNING [140330894071552] (hdr-config.cpp:364) In hdr_config::send_sub_preset_to_fw() - hw command failed: std::exception 06/10 14:36:18,900 WARNING [140330894071552] (hdr-config.cpp:364) In hdr_config::send_sub_preset_to_fw() - hw command failed: std::exception 06/10 14:36:26,315 WARNING [140330894071552] (hdr-config.cpp:364) In hdr_config::send_sub_preset_to_fw() - hw command failed: std::exception


4. Set `exposure` to `5000`:

06/10 14:36:31,191 WARNING [140330894071552] (hdr-config.cpp:364) In hdr_config::send_sub_preset_to_fw() - hw command failed: std::exception



It seems like the parameters can not be written to the camera. I think, I configured the HDR mode correctly, as the infra1 image is flickering as expected. I can also switch between over- and underexposed images via `sequence_id_filter` in `rqt_reconfigure`. Setting the exposure via launch file also works as expected, but I would prefer to set it dynamically.

Is this a known bug or perhaps some hardware issue on my side?

Ubuntu 20.04 
ROS noetic
librealsense: v2.54.2
D435i firmware: 5.15.0.2
running the latest commit from ros1-legacy branch.
Finn2708 commented 1 year ago

On further exploration, setting the exposure of the camera in HDR mode through the realsense-viewer results in a similar error:

hdr-config.cpp:371 - In hdr_config::send_sub_preset_to_fw() - hw command failed: std::exception
Setting Exposure to 5715

The new exposure is not updated immediately. The stream has to be toggled off and on again for changes to take effect. Is this the expected behaviour?

MartyG-RealSense commented 1 year ago

Hi @Finn2708 If you are using librealsense SDK version 2.50.0 then your camera firmware driver version should be downgraded to 5.13.0.50, as 5.15.0.2 is designed for use with SDK version 2.54.1. Using a firmware with an SDK version older than the one that it is recommended for can result in errors.

In regard to setting manual exposure dynamically, does the instruction below work for you please?

rosrun dynamic_reconfigure dynparam set /camera/stereo_module exposure 5715

Finn2708 commented 1 year ago

Hi @MartyG-RealSense, thanks for the quick reply. Looks like I have two versions of the SDK installed, which is probably not good. I will look into it.

When in HDR mode, rosrun dynamic_reconfigure dynparam set /camera/stereo_module exposure 5715 yields:

06/10 17:02:30,209 WARNING [139870281180928] (ds5-options.cpp:619) The control - Depth Exposure (usec) - is locked while HDR mode is active.

When I disable HDR, rosrun dynamic_reconfigure dynparam set /camera/stereo_module exposure 5715 works as expected and without any output in the terminal.

Finn2708 commented 1 year ago

Downgrading the device firmware to 5.13.0.50 resolved the issue. I am now able to dynamically set the exposure through rqt_reconfigure when in HDR mode.

Thanks :+1:

MartyG-RealSense commented 1 year ago

Excellent! Thanks very much for the update :)

MartyG-RealSense commented 1 year ago

Case closed due to solution achieved and no further comments received.

souljaboy764 commented 4 months ago

I was still having this issue in ROS1. It happened because I was trying to set the parameters via the json_file_path argument. 

 <include file="$(find realsense2_camera)/launch/rs_camera.launch">
    <arg name="align_depth" value="true"/>
    <arg name="tf_prefix" value="camera_right"/>
    <arg name="filters" value="decimation,hdr_merge,disparity,spatial,temporal"/>
    <arg name="enable_pointcloud" value="true"/>
    <arg name="clip_distance" value="1"/>
    <arg name="json_file_path" value="$(find data_recorder)/config/realsense_d405.json"/>
</include>

The JSON itself was generated by exporting the configuration from realsense-viewer for Depth Accuracy.

I also got an error when trying to change the preset in realsense-viewer. Something like

the control uvc image gain is locked while HDR mode is active

I could only do it by switching off the streaming and then changing the preset.

Setup Information: Camera Model D405 Firmware version 5.16.0.1 librealsense version 2.55.1 Ubuntu 20.04 ROS noetic

Based on this comment, After I downgraded the firmware to 5.13.0.50 and the librealsense version to 2.50.0, I was able to load the parameters using the json_file_path argument.

This unfortunately led to flickering in the image due the hdr_merge filter

ezgif-2-fc0d6f58b3

and the following warning in my console

 03/06 22:38:41,087 WARNING [140492795930368] (hdr-merge.cpp:38) HDR Merge filter cannot process frames because relevant metadata params are missing

If I immediately remove the hdr_merge filter from the launch file, then the image still flickers, but then once I unplug and replug the USB cable, the image comes normally.

I tried different USB ports on my PC as well but still had this issue

For now I have deactivated the hdr_merge filter since it doesn't affect the depth output significantly.

MartyG-RealSense commented 4 months ago

Hi @souljaboy764 When HDR is enabled in the librealsense SDK then the flickering can be stopped by configuring a post-processing filter called Filter By Sequence id. This filter is not included in the list of post-processing filters supported by the ROS wrapper though.

disparity_filter spatial_filter temporal_filter hole_filling_filter decimation_filter

So disabling HDR as you have done is likely to be the best course of action.