Questions about RealSense T265

[Hyjale]

Hello,

I noticed OpenVINS was tested with a RealSense T265 which has an accelorometer rate of 63hz. Was tracking quality sufficient enough even with such a low accelorometer rate?

Also, for use cases such as XR, where we want poses at 120+ FPS, what sensor would be recommended for use with OpenVINS?

[goldbattle]

Yeah, while I did not play and evaluate with it that extensively (did not compare to any groundtruth trajectory for example), it was reasonable pose output. It is a bummer that their max rate is only 63Hz which limits it application directly. I don't have any recommendation besides making your own sensor rig which our group has typically used Xsens IMUs which can provide 200-400hz IMU measurements. Another option is to handle this externally and use a pose prediction algorithm to try to increase the frequency (for example fit a split to the current history and predict future poses at a higher rate).

Is the 120fps what you need, or would something even higher be more helpful for XR applications? Do you have a feeling?

[Hyjale]

Yes, our system is running at 120 FPS (can be adjusted), but usually the current standard for XR applications is somewhere between 90 - 144 FPS.

[goldbattle]

Yeah, then I am not sure if there is any off-the-self system you can grab. If you do find one that would work well for XR applications, post it here and I can see if I can get and test out locally also, thanks!

[Hyjale]

Currently, our system is using the ZED-Mini sensor with an IMU rate of 500 Hz and image rate of 15 FPS with reduced exposure. These settings have personally worked the best with OpenVINS in which our experiments have shown us that while we are querying the pose at 120 FPS, having many more IMU samples than just one per query helps reduce jitter.

However, one concern is that ZED-Mini is rolling shutter. Do you have any experience with this particular sensor? If so, any suggestions to account for the rolling-shutter?

[goldbattle]

If the exposure is low, then the rolling shutter effect should be very minimal. You can also try inflating the pixel noise to handle this, but you can likely ignore it if the readout time is not very large you can run without considering rolling shutter. I think our lab has the ZED-mini, but I have not tried it, are you using their zed-ros-wrapper package?

Have you tried calibrating the readout time (I would be interested if you have a number on how much "rolling shutter" effect there is, if it is very small, then basically it is a global shutter camera). You can checkout one of my old PRs here which gives you the line-readout time: https://github.com/ethz-asl/kalibr/pull/261

[Hyjale]

Our system doesn't use ROS at all, instead I access the ZED SDK directly for cam-imu data as well as using our modified version of OpenVINS that is ROS-free. But, it shouldn't be too different in functionality if you use the zed-ros-wrapper package instead.

Thank you for pointing me to this! Will try it out and let you know.

[antithing]

@Hyjale Sorry to disturb you, would it be possible for you share the code you used to run the repo with the Zed camera? Did you use the SDK calibration data? Or kaliber? Thanks!