Any numbers for new models (open SP, ALIKED, SIFT)?

[ducha-aiki]

Hi,

I'd like to add some of this models to kornia if they are worth it. Given that you have trained them, I guess you have some numbers? If not, that's OK, just wanted to check.

[sarlinpe]

Hey @ducha-aiki, We've now released the LightGlue models trained for ALIKED & SIFT: https://github.com/cvg/LightGlue/commit/29f3e449efa1994758b8a16299d2816028dca65b These models are frankly very good - we've added results on MegaDepth and will add results on other datasets later (some ScanNet results in PR https://github.com/cvg/glue-factory/pull/25, others likely after the CVPR deadline). As you can see, ALIKED+LightGlue is better than SuperPoint+SuperGlue and pretty close to SuperPoint+LightGlue. For SIFT we need to double the number of keypoints to reach this accuracy (because the detector is much less repeatable).

For now we're not releasing the model trained for SuperPoint-Open because it still needs some more tuning.

While we're at it: it would be really great to have a proper implementation of DoG in kornia, ideally at the level of performance of vlfeat/GPUSIFT. The current implementation is really terrible (I know this was in your TODOs 2 years ago).

[ducha-aiki]

@sarlinpe thank you! I am surprised at ALIKED results tbh.

Regarding DoG in kornia - to be honest, while I see some value in it, and would like to finally fix that, I don't think it is going to be fast enough to compete with GPU-SIFT either alone or from colmap. And then, don't see much point there, given enormous memory footprint of the proper Gaussian scale space. Do you see any other arguments for it besides just completeness?

[sarlinpe]

• kornia is easier to install than pycolmap (GPU-SIFT requires building from source, we don't yet have the bandwidth to ship cuda-enabled wheels)
• when considering batching, a torch-SIFT could be faster than GPU-SIFT (which would make the training of LightGlue faster or at least more efficient)

Though I don't encourage anyone to use SIFT, except for data with strong in-plane rotations (where we don't have an optimal solution, yet).