naver / kapture-localization

Provide mapping and localization pipelines based on kapture format
BSD 3-Clause "New" or "Revised" License
269 stars 42 forks source link

Have you used any multicamera/sequence during localization like HLoc? #17

Closed AddASecond closed 3 years ago

AddASecond commented 3 years ago

multi-camera/sequence can provide significant improvement in visual localization. for example, Hierarchical Localization - SuperPoint + SuperGlue (multi-camera) have this function: https://github.com/cvg/Hierarchical-Localization

Have kapture-localization provided similar functions?

yocabon commented 3 years ago

Hi, right now we do not have multi camera pose estimation inside this repository.

I do have multi camera estimation scripts (similar to kapture_pycolmap_localize.py) with both colmap gp3p and MultiCameraPose but both are running on forks of the repositories and not suited to be shared as is.

For the colmap part, I'll integrate it here when this is merged https://github.com/mihaidusmanu/pycolmap/pull/13 (I actually cancelled my pull request about rig since Paul-Edouard Sarlin had a more advanced version)

For MultiCameraPose, I want this to be merged first (the MultiCameraPose changes are very similar so I'll do them afterwards) but I haven't heard anything back yet.

sarlinpe commented 3 years ago

FYI we will merge https://github.com/mihaidusmanu/pycolmap/pull/13 in the next days and release an integration with hloc.

yocabon commented 3 years ago

I added kapture_pycolmap_rig_localize.py and updated the Dockerfile to use the updated colmap / pycolmap.

It's not part of any pipeline right now, so to run it you have to import the colmap map obtained with the mapping pipeline with kapture_import_colmap.py.

the --input arg is map + query, with keypoints for all map + query images, matches for query -> map pairs for all pairs in the --pairsfile-path file; observations and points3d from the imported map

In --query, you don't need to have the reconstruction folder, only the sensors data is used.