No Quora & CQADupstack in leaderboard?

beir-cellar / beir

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.

http://beir.ai

Apache License 2.0

1.57k stars 190 forks source link

No Quora & CQADupstack in leaderboard? #57

Open Muennighoff opened 2 years ago

Muennighoff commented 2 years ago

Hey, why are Quora & CQADupstack excluded from the leaderboard?

nreimers commented 2 years ago

Hi @Muennighoff Thanks for reporting this. The spreadsheet was just an intermediate solution, not sure why Quora & CQA is missing.

I currently work on an updated leaderboard that will be launched soon, which will include these datasets

thakur-nandan commented 2 years ago

Hi @Muennighoff, Thanks for pointing it out. It has been there in my TODO list, More models started to pour in and I almost forgot about it. I will update the table soon and add the new numbers.

Kind Regards, Nandan Thakur