Online localization seldom succeeded

[bxc237]

Hi, I'm testing the ROVIOLI in Localization mode using an optimized loop-close map generated by maplab. But online localization seldom achieved (please check the video: https://www.dropbox.com/s/ob0f0fbz9etmb7a/online_localization_seldom_succeed.mp4?dl=0, and when I did this test, the scene and lighting conditions did not change comparing to the data that I used to generated the map). Should I try to tune those vi_map_landmark_quality parameters to get more landmarks? And parameters described in https://github.com/ethz-asl/maplab/wiki/Understanding-loop-closure have nothing to do with online localization, right? I would be really grateful if anyone could help me with this issue. Thanks.

[dymczykm]

Hi @BryceeeeeChen, thanks for such a detailed description of your problem.

We believe your issue might have multiple sources:

1. A significant part of your trajectory is non-overlapping and some parts only cover the environement when walking in the opposite direction. Traditional feature-based approaches do not handle very well such situations. Could you try to exactly follow the mapping trajectory to see if it helps?
2. Your environment seems to be a bit dark, which might cause unreliable feature detection. Additionally, it might cause motion blur. Could you try to rerecord the dataset with better lighting conditions and try again?

Additionally:

• The paramters in https://github.com/ethz-asl/maplab/wiki/Understanding-loop-closure influence online localization, so you can try to tweak them to obtain more localizations.
• I'm wondering how did you create the prior map. You said you've loopclosed it and optimized it. How did it look? Did you try to inspect it in RViz?

[bxc237]

@dymczykm Thank you for the reply! Firstly, for creating the map I was using in the video, I built four maps separately then merged and optimized it using these commands. maplab_commands.txt This is the map after running all commands that I attached. The map contained tons of landmarks from different viewpoints on the right side, so it should be good enough for localization, correct?

Actually, my knowledge of loop closure is very very limited. So in order to achieve relocalization, I have to strictly follow the mapping trajectory; it does not work like I see some landmarks in the same angle but three meters away from the same spot then hope the loopclosure engine tell me where am I, right? And what is the meaning of "find a balance between the loopclosure edges and the odometry edges (visual+inertial)" in the "Understanding loop closure" wiki page?...Thank you.

[bxc237]

And sometimes, when /debug_T_G_I_raw_localization published a red point (successful localization occurred), the trajectory published by /debug_T_G_I would not be corrected and relocated to the red point; the origin trajectory continued, is that normal?

[dymczykm]

@BryceeeeeChen sorry for a late reponse. Let me answer some of your questions:

1. Loopclosure performance is strongly depending on the matching viewpoint, but should tolerate walking 2-3 meters away from the mapped location in your case. Walking in an opposite direction is a much harder problem and I wouldn't expect it to work reliably.

2. Regariding the balance between the loopclosure and odometry edges. Please consult the switchable constraints paper (https://www.tu-chemnitz.de/etit/proaut/mitarbeiter/rsrc/IROS12-switchableConstraints.pdf). So you basically might need to tweak how stiff your localization constraints are, depending on your mapping scenario.

3. Yes, that's is normal in the current master. Depending on the estimator state, the jump might not happen. We will improve this behavior with a new release soon.

Your map looks quite OK, but I think we could improve it a bit. Regarding your commands - I would suggest to try a simplified (although slower) workflow and see if it improves anything:

sabu
aam
rtl 
optvi --ba_num_iterations 99
lc -lc_ransac_pixel_sigma 4 -lc_num_ransac_iters 500
lc -lc_ransac_pixel_sigma 4 -lc_num_ransac_iters 500
optvi --ba_num_iterations 99

could you try this and post the result? Optimizing and loopclosing twice is done on purpose.

[bxc237]

Hi @dymczykm, it took me some time to test the result of your suggestion since my old lab (the map I posted before) was getting redecorated. Therefore I used your suggested commands to generate another map in a new bigger scene. To test the localization, firstly, I was walking super slowly (around 0.3m/s) and holding the ZR300 at the same tilt angle in front of my chest that I obtained the data for generating localization map. In this condition, the relocalization performed very well. This is the video of the test, https://www.dropbox.com/s/hia8zpczsy2n0vk/new_scene.mp4?dl=0. The localization map generated by my own maplab commands also gave me similar performance. your suggested commands: my own commands: But if I held the ZR300 at the height of my waist and pitched it up just a little bit, the localization still seldom succeeded in both localization maps. This figure shows the problem in the map that generated by your commands. You can see the jump problem(or the trajectory could not be relocated) is serious as well. I'm also curious about what is the cause of the jump problem? Is a bug or the position of current camera is determined by more factors besides occurrence of localization? Thank you very much!

[dymczykm]

@BryceeeeeChen sorry for the late answer. Pitching the camera up shouldn't really influence the results, maybe it's a problem of lighting (e.g. the ceiling lighting makes the exposure too dark in the rest of the image?). Your maps looks consistent to be able to relocalize, that's for sure.

The jump problem is resulting from a possible bug when feeding an external pose to ROVIO. Have you tried if the Release Candidate branches help with that? The trajectory should be smooth there.

[bxc237]

Hi @dymczykm. Thanks. The release candidate did solve this issue. I just want to close this issue after you guys officially merge the PR. If you think this is unnecessary, we can close it now.