search

cvg / LightGlue

LightGlue: Local Feature Matching at Light Speed (ICCV 2023)
Apache License 2.0
3.39k stars 331 forks source link

Some fast traditional extractor #78

Open BoyceL opened 1 year ago

BoyceL commented 1 year ago

Hi developers! I'm using your wonderful work and the matching result is quite good, but I'm trying to use a much faster traditional extractor without network like ORB to deploy this method to a real-time application. I've noticed that you have provided some other extractors like SIFT, DISK, but they don't match my requirement a lot. So could you provide a lightglue model with ORB extractor? It will be sincerely appreciated.

sarlinpe commented 1 year ago

We don't have such models and we don't plan to train them in the near future, but you're welcome to do it yourself using https://github.com/cvg/glue-factory

ducha-aiki commented 1 year ago

@BoyceL you can train them probably very fast, but I would not expect ORB being good enough here. Also, if you are resource limited for training, check here, I have explored faster training options. https://github.com/cvg/glue-factory/issues/39

mattiasmar commented 10 months ago

@ducha-aiki Why do you expect ORB to not perform well?

ducha-aiki commented 10 months ago

First, because detections are not well spread. Run 500 orb points 500 superpoints and visualize. Second, because descriptor is much worse and less robust to SIFT

ducha-aiki commented 10 months ago

In fact, I quickly tried ORB training, it doesn’t train with default difficulty on homography pretraining, you have to make it easier at least

fettahyildizz commented 8 months ago

@ducha-aiki The reason why I would go with ORB instead of SIFT is, generally ORB runs much quicker and ORB might be the only option for humble environments. Extracting SIFT in real-time is not optimal for my case since I have little computational power in my robots. Did you try ORB training? If so, how are the results?

ducha-aiki commented 8 months ago

@fettahyildizz if you don't have a resources for SIFT or GPU-SIFT, I don't see how you can have resources for transformer-based matcher.

fettahyildizz commented 8 months ago

@ducha-aiki because every piece of resource matters in my case. BF Matcher doesn't give quality matches under certain conditions so in expense of run-time, I trade off with superior matcher quality. On the other hand extracting SIFT descriptor is 10x slower than ORB in most cases. So BF Matcher is not an option for most cases, I have to try LightGlue but I would really love to use LightGlue with ORB descriptors due to extracting descriptor time.

ducha-aiki commented 8 months ago

Then train it. I have tried a couple of times, but both fail, because it requires extensive hyper-parameter tuning, compared to training DoG-HardNet or DeDoDe, or any other thing I tried , and they just work.

fettahyildizz commented 8 months ago

@ducha-aiki I searched a bit but found noone trained ORB, so I was wondering in my mind why noone trained this before? Is it because of hard-work it requires, or it doesn't have great performance. Anyway, I will do my best to train when I have a chance.

ducha-aiki commented 7 months ago

I searched a bit but found noone trained ORB, so I was wondering in my mind why noone trained this before? Is it because of hard-work it requires, or it doesn't have great performance.

It is likely both. At least, for the hard work - I guarantee.