GrumpyZhou / image-matching-toolbox

This is a toolbox repository to help evaluate various methods that perform image matching from a pair of images.
MIT License
543 stars 80 forks source link

Further efforts to reproduce loftr hpatches #23

Open georg-bn opened 2 years ago

georg-bn commented 2 years ago

More changes related to #20 .

Running python -m immatch.eval_hpatches --config 'loftr' --task 'both' --rot_dir . --ransac_thres=3 --h_solver cv now yields

>>>>Eval hpatches: task=matching+homography method=LoFTR_outdoor_ds_noms scale_H=True rthres=3.0 thres=[1, 3, 5, 10] 
>>Finished, pairs=540 match_failed=0 matches=913.1 match_time=0.12s
==== Image Matching ====
#Features: mean=913 min=7 max=1000
#(Old)Matches: a=913, i=975, v=856
#Matches: a=913, i=975, v=856
MMA@[ 1  3  5 10] px:
a=[0.73 0.95 0.96 0.97]
i=[0.8  0.99 0.99 1.  ]
v=[0.66 0.91 0.94 0.95]

==== Homography Estimation ====
Hest solver=cv est_failed=0 ransac_thres=3.0 inlier_rate=0.93
Hest Correct: a=[0.65 0.88 0.92 0.95]
i=[0.82 0.98 0.99 1.  ]
v=[0.49 0.79 0.86 0.9 ]
Hest AUC: a=[0.4  0.67 0.76 0.85]
i=[0.56 0.81 0.88 0.94]
v=[0.25 0.53 0.65 0.77]

These numbers (at 3px) are actually higher than in the LoFTR paper. Take what you find useful from this pull request. I suppose you might want to wait with merging this until there is a reply here: https://github.com/zju3dv/LoFTR/issues/136