About webis-touche2020 qrels

beir-cellar / beir

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.

http://beir.ai

Apache License 2.0

1.55k stars 186 forks source link

About webis-touche2020 qrels #40

Closed guoday closed 3 years ago

guoday commented 3 years ago

Thanks for your great works. I notice that average number of docs per query changes from 49.9 to 19.0 in your latest version of the paper. To keep the same setting as your, what should I need to do?

Thanks.

thakur-nandan commented 3 years ago

Hi @guoday,

Nice observation! I was wondering when I will get this issue. I used the latest version of webis-touche2020 qrels which the dataset authors recently provided for the paper. I was waiting for the numbers on the leaderboard (exps which I need to rerun), and then upload the latest dataset.

Anyways, I will upload the latest dataset soon and update you back here once its done!

Kind Regards, Nandan Thakur

guoday commented 3 years ago

Thanks a lot for your reply. 😊

thakur-nandan commented 3 years ago

All done, you can find the latest dataset here: https://public.ukp.informatik.tu-darmstadt.de/thakur/BEIR/datasets/webis-touche2020.zip

Kind Regards, Nandan

guoday commented 3 years ago

Thanks for your quick action!!!

Best, Daya