search

gmberton / VPR-datasets-downloader

Automatic download VPR datasets in a standard format
MIT License
119 stars 18 forks source link

How to get msls-challenge? #15

Closed STSTERANDMOMO closed 2 months ago

STSTERANDMOMO commented 5 months ago

In many benchmarks,msls-challenge dataset is needed.How to get it?

jarvisyjw commented 5 months ago

I think it is an online challenge, you can check here for more info.

STSTERANDMOMO commented 5 months ago

I think it is an online challenge, you can check here for more info.

But the benchmark don't provide how to handle the msls-challenge dataset. I don't konw how to congnize the dataset.

gmberton commented 2 months ago

Hi, the challenge dataset does not have public labels and therefore we can't process it in the same way that we do with any other dataset, so we preferred not to include it in this list. Furthermore, when testing to submit the solution on the challenge's server, the test should be perform city by city (i.e. do retrieval for Athens's queries only within Athens's database), which would make it even more complex (and less consistent with the rest of the benchmark)