some question about rolling shutter setting parameter.

[chnhs]

Hello, thank you very much for opening up such great work. I'm looking at the details of the code recently. But at present, some problems have been bothering me. How do you deal with the rolling shutter problem? I don't see the relevant parameter settings in your code, such as "rolling_shutter_skew" or "rolling_shutter_readout_time". Thank you again, and look forward to your reply.

[oseiskar]

Hi!

This codebase does not include rolling-shutter compensation, which, in theory, is an issue on rolling-shutter cameras. However, in practice, the methods seems to perform quite well on rolling-shutter cameras too. For example, Vehicular experiments 4, 5 and 6 in the HybVIO paper use a device with rolling-shutter cameras (the ZED 2 system).

Also note that many of the benchmark datasets, such as EuroC, use global shutter cameras and they are also rather common in many modern AI/tracking cameras such as OAK-D, RealSense or Azure Kinect.

[chnhs]

@oseiskar Thank you very much for your reply. I read another part of the code yesterday. From the code, i see you added a "time_delay" item to the state estimation, so I wonder whether the estimation result of time_delay offsets the impact of rolling_shutter. I've used vins_mono to run the rolling shutter data before. The time delay is consistent with the impact of rolling shutter on state estimation, which will make the scale of trajectory larger or smaller. And I'd like to ask you, https://www.youtube.com/watch?v=c4BTrl0153g Can the effect inside be achieved with hybvio? Could you provide data related to rolling_shutter for us to verify? We look forward to your reply. Thank you again for providing such great work.

[oseiskar]

Good point. It is indeed possible that the time delay also helps to compensate the rolling shutter effect, in addition to its original purpose of tolerating incorrect/unknown/varying offsets between the IMU and camera clocks. However, it is also possible to turn off that feature using the estimateImuCameraTimeShift parameter and the method still works for rolling-shutter data.

Regarding the YouTube video, HybVIO should indeed have similar performance.

Could you provide data related to rolling_shutter for us to verify? We look forward to your reply. Thank you again for providing such great work.

We do not unfortunately offer such services. If you are interested in commercial collaboration, you can drop a message at https://www.spectacularai.com/#conctact