search

rpng / open_vins

An open source platform for visual-inertial navigation research.
https://docs.openvins.com
GNU General Public License v3.0
2.17k stars 636 forks source link

Intuition for deleting MSCKF features #275

Closed bhirschel closed 2 years ago

bhirschel commented 2 years ago

Hello, first of all: thank you for your great work! While trying around with different settings and new additions by myself, I noticed that the visualized MSCKF features are never the same from one frame to the next. This is because all successfully used features are flagged as "to_delete" and are subsequently removed from the database. By the way, I think you flag every feature as to_delete twice: the first time in the UpdaterMSCKF and the second time in VioManager. Anyways, can you give me an intuition on why that is necessary? In an inline comment you write that information shouldn't be reused, but why is that? I couldn't find any hint towards such a necessity in the original MSCKF paper from Li and Mourikis. I also tried to remove this line. Interestingly, it works perfectly well in a stereo-setup with deactivated online extrinsic calibration. With activated online calibration, the transformation of the cameras will be altered to a state where it doesn't make any sense. Also, in a monocular setup with deactivated calibration the estimation seem to quickly fail. I wonder why that happens and why it works in stereo. For my intuition this would be some kind of intersection between MSCKF and SLAM features, and it actually does create really nice looking feature tracks and long lasting feature tracking for many features, which I naively would consider a benefit. I am looking forward to your answer!

goldbattle commented 2 years ago

This comes from just classical state estimation. If you have a measurement, and have incorporated it into your recursive estimator (e.g., and EKF), then you shouldn't update a second time with the same measurement. This is what we call a "reuse of information" as you are adding information into the estimator which has already been used, thus will become overconfident. This holds for any type of measurement used to update an EKF.

bhirschel commented 2 years ago

Thanks for your answer! It helped me a lot understanding the issue. While thinking about it, I also noticed that for an EKF controlling eg a car, you would have measurements like velocity and turn-rate, which you also wouldn't reuse at a later time step. This approach must then be applied accordingly to systems that use visual measurements to update.