search

cvg / LightGlue

LightGlue: Local Feature Matching at Light Speed (ICCV 2023)
Apache License 2.0
3.15k stars 291 forks source link

Questions about SIFT+LightGlue #110

Open Uchan1996 opened 5 months ago

Uchan1996 commented 5 months ago

Thank you for your great work!

I would like to confirm the following things about your SIFT+LightGlue.

  1. Which type of SIFT is used for training of the published SIFT+LightGlue weight, OpenCV or PyColmap?
  2. Is there a performance gap between OpenCV SIFT and PyColmap one for the published SIFT+Lightglue weight.
  3. In the glue-factory implementation, PyColmap SIFT scores are calculated as the combination of DoG (all values are 1?) scores and scales while OpenCV scores as just DoG scores. How did you select top-k SIFT keypoints in training of the published SIFT+LightGlue weight and evaluation in your paper?
Phil26AT commented 5 months ago

HI @Uchan1996

  1. We use the GPU version of pycolmap for training (the wheels from conda are built with CUDA support).
  2. Yes there is a slight gap. pycolmap-SIFT is usually a bit better, mostly because it yields better detections. We still set opencv as the default because it is easier to install. I suggest heading over to gluefactory and benchmarking it on a few datasets.
  3. Yes this is a problem of pycolmap - it does not yield proper SIFT scores. Instead of doing the top-k selection in python, we rely on pycolmap to do it (with the option max_num_features.

duplicate of https://github.com/cvg/glue-factory/issues/63

Uchan1996 commented 5 months ago

Thank you for your detailed answers!

Uchan1996 commented 5 months ago

@Phil26AT Sorry, I have additional things to ask.

  1. When I use pycolmap (version>=0.5.0), the variable pred doesn't have the key keypoints_scores in gluefactory/models/extractors/sift.py. Then, the following error occurs in gluefactory/utils/export_predictions.py.

ValueError: Missing key {'keypoint_scores0', 'keypoint_scores1'}

  1. Do you perform nms filtering based on scales when using pycolmap SIFT? Currently, the variable pred doesn't have the key keypoint_scores if I use pycolmap (version>=0.5.0). So I have to set nms_radius to None since the function filter_dog_point need the argument pred['keypoint_scores'].

  2. Which version of pycolmap SIFT did you use for training and evaluation for the published weight.