search

beir-cellar / beir

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.
http://beir.ai
Apache License 2.0
1.55k stars 186 forks source link

create qrel.tsv file #47

Closed xiangpku closed 2 years ago

xiangpku commented 2 years ago

Hi, Nandan,

When we create the qrel.tsv file for custom preprocessed dataset, can it just innclude relevant documents, i.e., score=1 or non-relevant documents should also be included?

Cheers Xiang

thakur-nandan commented 2 years ago

Hi @xiangpku,

Only the relevant ones are necessary, i.e. with a score > 0. Everything else is automatically assumed with a score of zero if not present within the qrels.tsv file.

Hope it helps!

Kind Regards, Nandan Thakur

xiangpku commented 2 years ago

Great! Thanks again for your awesome work!

Xiang