Problem to get 60 hz depth of 640x480 resolution

[arindamsaha81]

|---------------------------------|------------------------------------------- | | Camera Model | D400} | | Firmware Version | 05.12.07.100 | | Operating System & Version | Ubuntu 16 | | Kernel Version (Linux Only) | 4.15.0-112-generic | | Platform | PC | | SDK Version | 2.38.1 | | Language | {C } | | Segment | {Robot } |

Hi,

I have just updated the SDK to 2.38.1 and firmware version to 05.12.07.100 to my D435I. I have built the SDK with CUDA support into my laptop with ubuntu 16.04 lts. I need 60 hz align_depth (640x480) for my application so I built with CUDA support. I am using the realsense-ros to publish the data to ros topics. I set fps for color image and depth image to 60. If I make align_depth = false the /camera/depth/image_rect_raw topic gives depth with 50 hz and if I make align_depth = true then the frequency of /camera/aligned_depth_to_color/image_raw is coming around 30 hz but if I make enable_infra1 = enable_infra2 = false then the frequency goes upto 46 hz. I am getting color image always at 60hz. Can you please help me to get 60 hz data. I am able to set 60 hz frequency to realsense-viewer and successfully recorded rosbag but the depth was not aligned in that case.

I have another observation; the accel and gyro gives 250 and 400 hz respectively but once I make unite_imu_method = linear_interpolation the imu frequency is about 400 hz which I used to get around 500 hz previously. Is this a normal behaviour?

Thanks in advance.

I am suddenly started getting frequency of around 15 hz only in all combination for both color and depth.. This started after the firmware update to 05.12.07.100.. totally confused..

[MartyG-RealSense]

Hi @arindamsaha81 Could you try disabling an RGB option called Auto Exposure Priority in your ROS launch file with rosparam instructions, please?

https://github.com/IntelRealSense/realsense-ros/issues/1349#issuecomment-681695259

If Auto Exposure is enabled and Auto Exposure Priority is disabled then the FPS should be forced to be constant.

Regarding the IMU rate, I have heard of cases where higher rates have been achieved. Usually it has been by accident and attempts to replicate it have failed. So '400' should be the rate that is considered to be correct.

[arindamsaha81]

Hi @MartyG-RealSense

Thanks for the quick reply. I have changed as per the comment https://github.com/IntelRealSense/realsense-ros/issues/1349#issuecomment-681695259 and https://github.com/IntelRealSense/realsense-ros/issues/1354#issuecomment-681126057 but nothing is working for me. In all cases the frequency is 15 hz.

[MartyG-RealSense]

It is recommended that the RealSense ROS wrapper version is matched as closely as possible to the librealsense version recommended in the release notes for a particular wrapper version. For example, wrapper 2.2.16 is recommended for librealsense version 2.37.0 (the release of new ROS wrappers is not synchronized with the release of new librealsense versions), whilst ROS wrapper 2.2.15 is recommended for use with librealsense 2.36.0.

It has been known for a wrapper to work with a newer librealsense version than recommended if the new SDK is not too different to the previous one. In general though, it is best to adhere to the match-up recommendations.

Which ROS wrapper version are you using with 2.38.1 please?

[arindamsaha81]

Initially my SDK version was 2.37.0, firmware version was 05.12.06.00 and ROS wrapper version was 2.2.16. Today I have updated the SDK to 2.38.1 and firmware to 05.12.7.100 but ROS wrapper version is still 2.2.16.

[arindamsaha81]

Today morning (in daylight) I found the frequency is back to 60 hz for color image and 60 hz for align_depth (even if I enable the Auto Exposure Priority) where enable_infra1 = enable_infra2 = false. Yesterday when it dropped to 15 hz, the testing was under indoor light at night time (Auto Exposure Priority was disabled). So I guess the lower frequency because of the lighting condition and disabling Auto Exposure Priority did not work properly.. Are these assumptions correct?

If I make enable_infra1 = true and enable_infra2 = false then the align_depth becomes 30 hz. But once I disable both infra1 and infra2 then I guess depth is not good. So how can I get good align_depth at 60 hz?

[MartyG-RealSense]

I have seen a few cases in the distant past where FPS was apparently dragged down (e.g from 30 to 17) when a camera was used in darkened lighting conditions.

As an alternative to using Auto Exposure = true and Auto Exposure Priority = false, if Auto Exposure is disabled then you can enforce a constant FPS by setting manual exposure in a certain range. It is complicated though, as an exposure value that is set too low or too high can cause FPS to fall or increase. The link below provides more details about the maths of it.

https://github.com/IntelRealSense/librealsense/issues/1957#issuecomment-400715579

The effect of manual exposure values on FPS when aligning depth and color at 60 FPS is also discussed here:

https://support.intelrealsense.com/hc/en-us/community/posts/360037286634-Strange-FPS-when-enable-both-color-and-depth-streams

Doronhi the RealSense ROS wrapper developer has said about disabling infra2: "It will have no effect on the depth quality. It only disable the infra2 images' transmission via the USB port. They are still taken and the depth is calculated using them inside the device".

From ROS wrapper 2.2.15 onward, infra is disabled by default, so you should configure the true / false state of infra1 and infra2 if you need to use infra.

Regarding speed optimizations, I would add that if you are building librealsense from source code with CMake then the term below should ideally be included so that optimizations can be taken advantage of (the optimizations are already enabled if you build librealsense from packages).

-DCMAKE_BUILD_TYPE=release

[arindamsaha81]

I am successfully set the required parameters from realsense_viewer. I have unset the "Enable Auto Exposer" option in "Stereo Module" and also unset the "Auto Exposure Priority" option in "RGB Camera" in realsens_viewer. In the ROS launch file I changed enable_infra1 = false and enable_infra2 = false. This allows me to get a fixed 60 hz rgb and depth images in all lighting conditions. My current exposer value is showing as 8500.000 in the realsense_viewer, which unit it is? millisecond or microseconds?

Regarding speed optimization I have already taken care of building the SDK in release mode.

[MartyG-RealSense]

Settings made in librealsense programs such as the RealSense Viewer have no effect on the camera's performance in the RealSense ROS wrapper. Functions such as Auto Exposure Priority should be set from within the ROS wrapper if you are using those functions in ROS.

Changing Auto Exposure Priority in the Viewer will do nothing to the AE Priority status in ROS. You should change the true / false status of Auto Exposure Priority in the ROS wrapper by using rosparam instructions in the launch file, as mentioned earlier in this case:

https://github.com/IntelRealSense/realsense-ros/issues/1349#issuecomment-681695259

[arindamsaha81]

Thanks for the clarification.

I was doing a mistake last time when I tried to set "Auto-Exposure" and "Auto-Exposure Priority", I put "/camera/rgb_camera/rs435_color_auto_exposure_priority: false" but I renamed the camera as realSense_d435i from the command as "roslaunch realsense2_camera rs_aligned_60HzDepth_imu.launch camera:=realSense_d435i serial_no:=843112072984 filters:=spatial,temporal". So I set rosparam as "/realSense_d435i/rgb_camera/rs435_color_auto_exposure_priority: false". In this case I get 48 fps of align_depth in day lighting condition. But when I set "/realSense_d435i/rgb_camera/enable_auto_exposure: false" as explained in https://github.com/IntelRealSense/realsense-ros/issues/1354#issuecomment-681126057 , I am getting below error after launching the launch file. "[ERROR] [1599113437.454055875]: hwmon command 0x75( 0 1df 0 27f ) failed (response -6= Invalid parameter) [ WARN] [1599113437.984807014]: Hardware Notification:Depth stream start failure,1.59911e+12,Error,Hardware Error" Is this error could be for wrong manual exposure value already set? Because I am unable to set the manual exposure through rqt_reconfigure. I dont see any place to change the exposure value. The attached image shows the screen shot of rqt_reconfigure.

[arindamsaha81]

I have unset the option for "Enable Auto Exposer" under "RGB Camera" in realsense_viewer, set exposure value to 15.000 and did other necessary changes like resolution and FPS. Save this configuration into a json file. Load this json file through the launch file of the ROS wrapper. But I still get the same error as mentioned before.. screen shot is attached.

[arindamsaha81]

I am able to set the exposure value manually as explained in https://github.com/IntelRealSense/realsense-ros/issues/1245#issuecomment-649882792 I can see the exposure value changes of the RGB image in rviz. But that does not change the align_depth frequency. I am checking the align_depth frequency using commands "rostopic hz /realSense_d435i/aligned_depth_to_color/image_raw" which always shows about 47 to 48 hz.

Do I need to disable the "Auto-Exposure" for only "rgb_camera" or "stereo_module" also?

[MartyG-RealSense]

You can try disabling depth auto-exposure with a rosrun command:

rosrun dynamic_reconfigure dynparam set realsense2_camera_manager rs435_depth_enable_auto_exposure 0

[arindamsaha81]

Below output I got by running command "rosrun dynamic_reconfigure dynparam get /realSense_d435i/stereo_module"

{'laser_power': 150.0, 'groups': {'laser_power': 150.0, 'parent': 0, 'emitter_always_on': False, 'global_time_enabled': True, 'inter_cam_sync_mode': 0, 'gain': 16, 'groups': {}, 'id': 0, 'output_trigger_enabled': False, 'exposure': 8500, 'name': 'Default', 'parameters': {}, 'emitter_enabled': 1, 'enable_auto_exposure': True, 'error_polling_enabled': True, 'state': True, 'visual_preset': 0, 'emitter_on_off': False, 'type': '', 'frames_queue_size': 16}, 'output_trigger_enabled': False, 'emitter_always_on': False, 'enable_auto_exposure': True, 'global_time_enabled': True, 'inter_cam_sync_mode': 0, 'visual_preset': 0, 'error_polling_enabled': True, 'gain': 16, 'emitter_enabled': 1, 'frames_queue_size': 16, 'emitter_on_off': False, 'exposure': 8500}

This says the exposure value is set to 8500 when enable_auto_exposure is true. This value is in micro seconds? The value looks quite small if it is micro seconds but it is very very big for miliseconds. So I am confused.

[MartyG-RealSense]

8500 is the default value that the depth manual exposure is set to in the Stereo Module controls in the RealSense Viewer program, so it is likely to be an appropriate value to be used here.

RGB exposure tends to use much smaller values, such as 100 or 150.

Exposure units are in microseconds.

[MartyG-RealSense]

Case closed due to no new comments received.