search

cvg / Hierarchical-Localization

Visual localization made easy with hloc
Apache License 2.0
2.96k stars 551 forks source link

Superpoint + superglue w/ hloc give 90 degree camera orientation error #362

Open sfm-user-1 opened 5 months ago

sfm-user-1 commented 5 months ago

Hi, thanks for making this repo and the notebooks.

I tried out the superpoint+superglue+colmap notebook here https://github.com/cvg/Hierarchical-Localization/blob/master/pipeline_SfM.ipynb

using the 35-image dataset here: http://vision.maths.lth.se/calledataset/de_guerre/de_guerre.zip

spsg_vs_sift_colmap

It seems superpoint+superglue w/ HLOC estimates most camera orientations incorrectly by 90 degrees error about the +z camera axis (roll). I don't see any such issue with vanila COLMAP. I wonder if you could try reproducing or investigating this result?

Thanks again!

sfm-user-1 commented 4 months ago

Hi, a gentle bump on this. 90 degree orientation error for most cameras seems like it is a bug in the hloc source code. Thanks!

sarlinpe commented 4 months ago

This indeed looks like a problem, I'll investigate once I find the time. Thank you for reporting this!

sarlinpe commented 4 months ago

Found the culprit - PR https://github.com/cvg/Hierarchical-Localization/pull/373 solves this bug. Unfortunately SuperPoint+SuperGlue cannot match 90-degree rotated images, so for now hloc registers only 26/35 images (those in portrait mode). Fixing this is pretty high on our priority list. For now I suggest baking the EXIF rotation into the images by reading+writing them with OpenCV:

import cv2
for p in input_dir.iterdir():
    cv2.imwrite(str(output_dir / p.name), cv2.imread(str(p)))

This prevents COLMAP from sharing intrinsics between rotated images but the negative impact should be negligible.