search

cvg / glue-factory

Training library for local feature detection and matching
Apache License 2.0
756 stars 98 forks source link

Adds evaluation on challenging WxBS and EVD datasets #52

Open ducha-aiki opened 10 months ago

ducha-aiki commented 10 months ago

Closes https://github.com/cvg/glue-factory/issues/35

Compared to original papers, I have added the image resize to be consistent with gluefactory evals, and to make things easier for learned features.

Also turned out, that big homography changes in EVD are harder than appearance changes in WxBS.

EVD

mAA @ 1/5/10/20 px

Methods (2K features if not specified) PoseLib
SuperPoint + SuperGlue 0.0 / 5.4 / 10.1 / 11.7
SuperPoint + LightGlue 0.0 / 5.5 / 10.4 / 11.8
SIFT (4K) + LightGlue 0.0 / 3.8 / 5.2 / 10.0
DoGHardNet + LightGlue 0.0 / 5.5 / 10.5 / 11.9
ALIKED + LightGlue 0.0 / 5.4 / 12.4 / 16.2
DISK + LightGlue 0.0 / 0.0 / 6.9 / 10.1

WxBS

mAA @ 1/5/10/20 px Methods (2K features if not specified) PoseLib
SuperPoint + SuperGlue 13.2 / 39.9 / 49.7 / 56.7
SuperPoint + LightGlue 12.6 / 34.5 / 44.0 / 52.2
SIFT (4K) + LightGlue 9.5 / 22.7 / 29.0 / 34.2
DoGHardNet + LightGlue 10.0 / 29.6 / 39.0 / 49.2
DoGHardNet (4K) + LightGlue 16.3 / 41.6 / 51.4 / 59.7
ALIKED + LightGlue 18.7 / 46.2 / 56.0 / 63.5
DISK + LightGlue 15.1 / 39.3 / 48.2 / 55.2
ducha-aiki commented 9 months ago

Thank you for review, I will make changes shortly, where possible.

Regarding the "libpng warning: iCCP: known incorrect sRGB profile" - that is an issue with the images themselves, I will check if anybody still have an access to that server to update them.