search

AoxiangFan / EifficientDeterministicSearch

Implementation of ECCV 2020 paper "Geometric Estimation via Robust Subspace Recovery" and IEEE TPAMI 2021 paper "Efficient Deterministic Search with Robust Loss Functions for Geometric Model Fitting"
GNU General Public License v3.0
18 stars 5 forks source link

results on image matching benchmark[63] #1

Open ducha-aiki opened 3 years ago

ducha-aiki commented 3 years ago

Hi,

First, thank you for finding our benchmark useful!

However, I need to point out, that there is a significant mismatch between our benchmark setup and the setup applied in your paper here https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9528069

Specifically,

All compared methods are used with 0.99 confidence and maximum iterations of 10,000 to balance efficiency and accuracy." image image

is not a correct way of comparing the estimators, because they have different complexity and meaning of iteration. 1 iteration of MAGSAC makes more computation than 5 RANSAC iterations. Therefore in the paper and the challenge, we use the following setup:

challenge: https://www.cs.ubc.ca/research/image-matching-challenge/2021/submit/ image

paper: "Optimal values for Γ can vary drastically, from 10k for MAGSAC to 250k for PyRANSAC." image

AoxiangFan commented 2 years ago

Hi,

First, thank you for your attention to our paper! Also thank you for proposing the benchmark. It is really an important contribution to the community.

I am sorry that we have not noticed the recommended benchmark setup for robust estimators, since the experiments were finished a long time ago. I am now preparing a release of the python implementation of our code and I will try to redo the experiment to see the comparison results in the new setup.

Hi,

First, thank you for finding our benchmark useful!

However, I need to point out, that there is a significant mismatch between our benchmark setup and the setup applied in your paper here https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9528069

Specifically,

All compared methods are used with 0.99 confidence and maximum iterations of 10,000 to balance efficiency and accuracy." image image

is not a correct way of comparing the estimators, because they have different complexity and meaning of iteration. 1 iteration of MAGSAC makes more computation than 5 RANSAC iterations. Therefore in the paper and the challenge, we use the following setup:

challenge: https://www.cs.ubc.ca/research/image-matching-challenge/2021/submit/ image

paper: "Optimal values for Γ can vary drastically, from 10k for MAGSAC to 250k for PyRANSAC." image

ducha-aiki commented 2 years ago

Sounds great! Feel free to ping me if you need any help with running the benchmarks, or something about the benchmark is unclear.

AoxiangFan commented 2 years ago

Hi,

Sorry for the delay but we have now released the Python implementation of our EAS method.

The aforementioned experiment is re-conducted in renewed setup and the result is given below (with SOSNet as descriptor).

Anyway, the code is released in this repo and it should be easy to test the method now, probably using more datasets as in your blog if you are interested :).

Please feel free to ask if you have some other problems regarding our work.

ransac-rt-example