GrumpyZhou / image-matching-toolbox

This is a toolbox repository to help evaluate various methods that perform image matching from a pair of images.
MIT License
543 stars 80 forks source link

Reproduce loftr hpatches (almost) #22

Closed georg-bn closed 2 years ago

georg-bn commented 2 years ago

Potential fix for #20 . Running

python -m immatch.eval_hpatches --config 'loftr' --task 'homography' --root_dir . --ransac_thres=3 --h_solver cv

yields

>>>>Eval hpatches: task=homography method=LoFTR_outdoor_ds rthres=3.0 thres=[1, 3, 5, 10] 
>>Finished, pairs=540 match_failed=0 matches=2599.4 match_time=0.12s
==== Homography Estimation ====
Hest solver=cv est_failed=0 ransac_thres=3.0 inlier_rate=0.86
Hest Correct: a=[0.64 0.87 0.92 0.95]
i=[0.81 0.98 0.99 1.  ]
v=[0.49 0.77 0.85 0.91]
Hest AUC: a=[0.36 0.65 0.75 0.84]
i=[0.48 0.79 0.87 0.93]
v=[0.25 0.52 0.64 0.76]

which should be compared to the values reported in the paper: Hest AUC: a=[?, 65.9%, 75.6%, 84.6%]. There they only use the top 1k matches, which I haven't tried yet.

Most important changes:

I don't know if the changes I made break something non-LoFTR, non-HPatches related, but they probably do so you will want to rewrite some of the changes.

GrumpyZhou commented 2 years ago

@georg-bn ,

Thank you very much for the patch, and found the HPatch bug ! So most of the commits I am good with. The only a bit concerning one is d8c0d0a. But I will first merge your request and change that