New Images - Githubissues

cvg / Hierarchical-Localization

Visual localization made easy with hloc

Apache License 2.0

3.14k stars 581 forks source link

New Images #22

Closed PKehnel closed 3 years ago

PKehnel commented 4 years ago

Hey, thanks a lot for the amazing toolbox. I got the project working on a custom dataset, now my goal would be to use this model and the matches / feature files to localize an new image, that was not part of the original dataset.

Any ideas how to approach this, obviously I get an Object missing error, when just using the previous matches /feature files. Hope this is not out of scope.

sarlinpe commented 4 years ago

Sure, here is how I would do it:

Run extract_features on the folder of query images, pointing to the same feature file as the reference images;
Generate a list of localization pairs between each query and one or several references images, e.g. using image retrieval with DIR, prior poses from local SLAM tracking or GPS, or any other mean -sorry this step is not integrated yet but hopefully soon.
Run match_features using this list of pairs, pointing to the same match file as the reference images;
Assuming that you have intrinsics for the queries, generate a list of queries with intrinsics, as parsed here. You will need at least the focal length, which you could get from EXIF metadata.
Run localize_sfm :)

PKehnel commented 4 years ago

Thanks for the advice, this seems to be similar to your approach from the dev branch for 4Seasons. Will try it out! Goal is to run it in real time in a private ar application. Lets see.

PKehnel commented 3 years ago

Loc pairs: Since DIR on a custom dataset is not really possible for me (Issue: Unfortunately, You need to implement multi-staged backpropagation in your self.)

Would it also be possible to generate the localization pairs for the query and some references image, via similarity in the feature files? Using the descriptors and the scores in a simplified way and storing everything in a Annoy or some other similarity search database?

sarlinpe commented 3 years ago

DIR provides pretrained models that should generalize well to multiple environments - retraining should not be needed. Alternatively, you could indeed perform image retrieval with local features using something like Bag-of-Words. ASL-Feat has an example.