question about slow motion mode

[CanCanZeng]

Hi, I'd like to ask that if there is a motion mode that open_vins hard to handle? I'm facing an odd problem. My data is recored on a hand held device, open_vins runs well on most situation, but I find that if I hold device very stable and moving very slowly, open_vins will dift after some time.

I'm more familiar with optimization based vio algorithms such as vins-mono, it handles slow moving mode very easily, since if moving very slowly, the second latest frame instead of the oldest frame will be marginalized, so the siliding window will still preserve good structure, so the algorithm will not drift.

How can filter based algorithm handle this situation? I tried to switch on zero velocity detection, but it still drifted away, since the camera was still moving.

[WoosikLee2510]

Simply speaking, the reason for openvins suffering from slow motion is because of the front-end (feature extraction, matching, etc) not the back-end (estimator). If you move slow, feature triangulation fails that it gives wrongful info to the estimator that openvins drifts.

[CanCanZeng]

Thank you for your reply @WoosikLee2510 . You are right, the drift is caused by the front end, not the back end. Could you give some suggestion on over come my problem? Some times I need to move very slowly, so I may need to modify the code for this situation.

[WoosikLee2510]

You can try Vinsmono's thresholding method. When you get a new image, 1) compute propagation 2) check if motion was over the threshold 3-1) actually propagate the state if is is good, 3-2) cancel propagation and discard image if it is bad, 4) update with image if propagation is not failed.

You may also need to change the image tracking and propagation order as current system adds the new image to the database before doing the propagation.

[CanCanZeng]

Hi, thank you for your advice! @WoosikLee2510 When I was trying to fix this problem, I find that the v2.3 version would not fail for my data. The commit I test is c6f7e947e82485873495123a1841bd57f8041a33 and 6d65547d5a2399803af6c50a4e590f1fb7bc1b76, the earlier version is apparently more robust to slow motion mode and fast pure rotation motion mode. I'm still digging into this phenomenon, and I will inform you as I have further findings.

[CanCanZeng]

Hi @WoosikLee2510 @goldbattle , can you give me an email address that I can provide a sample data to show the difference between v2.3 and v2.5? I notice that there is also another issue about the performance degeneration of the newer version https://github.com/rpng/open_vins/issues/192, maybe you need more challenge data for regression testing.

[WoosikLee2510]

Sure. My email address is woosik@udel.edu.

[goldbattle]

Please try the latest release. If this is still an issue, feel free to reopen, thanks!

[CanCanZeng]

Hi @goldbattle , I tried the latest release v2.6.2, it solved my problem in most cases. But there are still some situations that open_vins will fail, mainly for slow motion initialization. I tried standing still at the beginning, but since the camera is hand held, there is no guarantee of complete stillness. In this case, open_vins will dirft away whether I use static initialization or dynamic initialization.

But I find that if I set the tracking frequency to 11 Hz, the system runs very well. I don't know why, do you have any idea? And actually I cannot always set tracking frequency to 11 Hz, since the camera may shake or change orientation suddenly. Is there any way to dynamically change the tracking frequency? maybe we can use disparity of feature point and imu. (I tried open zero-velocity update, the result is not as good as slow down the tracking frequency)

[CanCanZeng]

I uploaded a data and configuration file to google drive https://drive.google.com/drive/folders/1MYbd1SKIIMDfx44SCBtxkvc5gV4YviTr?usp=sharing wish you can check it when you have time

[goldbattle]

In order to initialize the system, some disparity is needed to recover the depth of the features (if you have stereo then you can initialize directly). Typically, we threshold the beginning such that we only initialize after sufficient parallax.

Reducing the tracking frequency enforces that the 11 clone states actually span double the amount of time (as compared to tracking at 20hz). This of course reduces the amount of information and thus is bad in dynamic motion, but is good for reducing computational cost and improving disparity between frames. For example, VINS-Mono typically uses 10hz feature tracking in their system.

[CanCanZeng]

@goldbattle Yes, I know that vins-mono use 10 Hz feature tracking. But there is a small difference between the current implementation of open_vins and vins-mono: vins-mono publish feature message after optflow tracking, so it can work well even in fast moving mode. I also tried to realize it in open_vins, I tried to early return after feature tracking (around here https://github.com/rpng/open_vins/blob/cd866b5ba58f1b59f0d6983c0a379753d25d1308/ov_msckf/src/core/VioManager.cpp#L261), but it does not work, I may go deeper to see why it does not work.

[goldbattle]

True, very good point, I should probably make this an option.

On Tue, Aug 16, 2022 at 12:09 PM CanCanZeng @.***> wrote:

@goldbattle https://github.com/goldbattle Yes, I know that vins-mono use 10 Hz feature tracking. But there is a small difference between the current implementation of open_vins and vins-mono: vins-mono publish feature message after optflow tracking, so it can work well even in fast moving mode. I also tried to realize it in open_vins, I tried to early return after feature tracking (around here https://github.com/rpng/open_vins/blob/cd866b5ba58f1b59f0d6983c0a379753d25d1308/ov_msckf/src/core/VioManager.cpp#L261), but it does not work, I may go deeper to see why it does not work.

— Reply to this email directly, view it on GitHub https://github.com/rpng/open_vins/issues/225#issuecomment-1216848414, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQ6TYXUDV5CCF2C7BYTS3LVZO4JZANCNFSM5NWIQBIA . You are receiving this because you were mentioned.Message ID: @.***>