search

MubarizZaffar / CoHOG_Results_RAL2019

This repository contains the results for our publication titled "CoHOG: A Real-time, Light-weight and Training-free Visual Place Recognition Technique for Loop-closure in Changing Environments"
12 stars 0 forks source link

Results reproducibility #1

Closed simonefelicioni closed 2 years ago

simonefelicioni commented 2 years ago

Hello,

I am trying to reproduce the results on the Gardens Point dataset stated in the paper, but I cannot get them. May you clarify how you computed the precision metrics?

Thanks, Simone

MubarizZaffar commented 2 years ago

Hi Simone,

I am not sure what you mean exactly here about not being able to reproduce the results but the precision in Table 1 of the paper is Precision at 100% Recall, which is essentially the total number of correct matches divided by the total number of query images in the dataset.

There is also the part about ground-truth tolerance in the Gardens Point dataset, for example since query image k seems to have reference images from k-2 to k+2 as coarsely the same place they are considered as correct matches. Maybe you could also see results at k-1 to k+2 and k-3 to k+3 as well. I do not remember exactly what range I had used back then but it was based on earlier papers. People have used different ranges for this in their works (which has major impact on precision), which is something I also discussed in my VPR-Bench paper.

Regards Mubariz

simonefelicioni commented 2 years ago

Thanks for your reply! I could not find any reference to the tolerance, so I thought there were some issues on the code I'm using. Now I've just checked out the paper you mentioned and I've finally obtained your same results.

Thank you, Regards Simone

MubarizZaffar commented 2 years ago

Great, I am going to close this issue now. Good luck with your research.