search

QVPR / Patch-NetVLAD

Code for the CVPR2021 paper "Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition"
MIT License
510 stars 72 forks source link

How to get the value of Extended CMU Seasons? #71

Closed MAX-OTW closed 1 year ago

MAX-OTW commented 1 year ago

Hello @Tobias-Fischer @StephenHausler ,I really appreciate for your work. I have a question, the Supplementary Table 2 shows that the result of Extended CMU Seasons of Patch-NetVLAD: Urban:19.2 / 48.0 / 97.2; Suburban:8.2 / 28.8 / 97.0; Park:9.5 / 34.9 / 94.3

But in the Table 1,the result is : 11.8/36.2/96.2

Is it weighted in some way? How is it converted? I am always looking forward to your kind response. Best regards.

Tobias-Fischer commented 1 year ago

Hi @MAX-OTW - as far as I can remember, the main paper contains the weighted average (by number of images) of what is shown in Supp. Table 2 (where the dataset is split by condition). I hope this helps :)

MAX-OTW commented 1 year ago

@Tobias-Fischer ，hi, thanks for your reply. But I still can't find any hints in the main paper and supplementary material: (. By the way, how do you convert the results of RobotCar Seasons v2? Could you please tell me the specific weighted average of these two datasets? I am always looking forward to your kind response. Best regards.

Tobias-Fischer commented 1 year ago

Do you have the datasets at hand? It should be easy to calculate the weighted mean given the number of images per condition. Please see https://github.com/QVPR/Patch-NetVLAD/issues/45 re RobotCar v2 - @StephenHausler is still looking into this (he has moved to a new position, so unfortunately only has limited time to look into this).

MAX-OTW commented 1 year ago

Hello, @Tobias-Fischer -I just checked the Extended CMU Seasons the data set (note:it comes from the site https://www.visuallocalization.net/). To my surprise,weighted by the results provided by Supplementary Table 2 in the paper, the results obtained were different from those in Table 1.So I have the following two questions：

1、How many pictures are there in Urban, Suburban and Park in the Extended CMU Seasons data set used in this paper? Could you please provide the name of the specific pictures used (three txt files are formed, including the path of these pictures)?

2、Were the results of the two datasets in the paper(RobotCar Seasons v2 and Extended CMU Seasons)tested on this site(https://www.visuallocalization.net/)? If so, why don't see it on the benchmark in the website (https://www.visuallocalization.net/benchmark/)? Or did you get it some other way?

I am always looking forward to your kind response. Best regards.

Tobias-Fischer commented 1 year ago

Hi, Park: 32322 Suburban: 43224 Urban: 31200 Please check out the Extended CMU readme to figure out the file paths based on the slices.

With these numbers, the weighted average is exactly as written in the paper, e.g. (323229.5+432248.2+31000*19.2)/(32322+43224+31000)=11.8

The results were obtained on the benchmark, but we did not make the results public.