Open mstiehm-NF opened 7 months ago
Hi, I think this might be bandwidth related, by default in RGBD mode Stereo is scaled up for better alignment with RGB frame (and its FPS is based on left/right streams). This can be overridden with i_height/i_width params, or you can shrink down rgb frames using ISP scaling. If that does not improve things, you can also try using low_bandwidth mode for the sensor streams. More information on latency can be found here.
I have modified my yaml to the following:
/oak:
ros__parameters:
camera:
i_calibration_dump: true
i_enable_imu: true
i_enable_ir: true
i_pipeline_type: RGBD
i_publish_tf_from_calibration: true
i_tf_imu_from_descr: 'true'
i_usb_speed: SUPER_PLUS
stereo:
i_publish_topic: true
i_reverse_stereo_socket_order: true
i_publish_synced_rect_pair: true
i_publish_left_rect: true
i_publish_right_rect: true
i_update_ros_base_time_on_ros_msg: false
i_align_depth: true
i_resolution: '400P'
i_low_bandwidth: true
i_low_bandwidth_quality: 1
imu:
i_acc_cov: 0.0
i_acc_freq: 250
i_batch_report_threshold: 5
i_enable_rotation: true
i_get_base_device_timestamp: false
i_gyro_cov: 0.0
i_gyro_freq: 200
i_mag_cov: 0.0
i_max_batch_reports: 10
i_message_type: IMU
i_rot_cov: 0.0
i_rot_freq: 200
i_sync_method: LINEAR_INTERPOLATE_ACCEL
i_update_ros_base_time_on_ros_msg: false
left:
i_height: 400
i_width: 640
i_fps: 117.0
i_low_bandwidth: true
i_low_bandwidth_quality: 1
right:
i_height: 400
i_width: 640
i_fps: 117.0
i_low_bandwidth: true
i_low_bandwidth_quality: 1
rgb:
i_resolution: '1080P'
i_isp_den: 3
i_isp_num: 2
i_output_isp: true
i_fps: 30.0
i_low_bandwidth: true
i_low_bandwidth_quality: 1
i_publish_topic: true
use_sim_time: false
And I am still only able to get max 30hz from my left and right images and about 30hz on my IMU. I am at least getting the 30hz on the rgb image though. Is there any other way to get the full 90hz out of the images and 200hz from the IMU?
Hi, sorry for the delay, as mentioned this is probably related to bandwidth, for rectified stereo streams you need to set those parameters separately, for example i_right_rect_low_bandwidth_quality: 50
(full yaml file can be found here)
Okay, I'll try that now. Are the IMU noise values on your Isaac PR what I should be using for the BMI270?
I was also able to achieve better results (60fps on the rect streams) by turning the nn off. Still only 60hz on the IMU data though. I'm wondering if it's a processing bottleneck, not just a pure bandwidth problem.
Thanks! ~Matt
I have been trying to use the Oak-D Wide and Oak-D Lite in RGBD mode for use in vslam using the PR here
I have noticed that the camera is not able to publish at the requested rates with either the Depth or RGBD pipelines.
My RGBD yaml is:
All the topics, including the IMU are not able to reach the desired rates. The USB is in SUPER mode.
Any help here would be appreciated.