Closed jordane95 closed 2 years ago
Hi @jordane95, thank you for your questions!
test.tsv
comes from the passage ranking task in TREC-DL 2019 (https://microsoft.github.io/msmarco/TREC-Deep-Learning-2019.html#passage-ranking-dataset).train.tsv
only will be binary and 1 as we only care about relevant passages (i.e. with score = 1) while training. However in evaluation for a few datasets, we may have multiple levels of relevancy, i.e. a score of 3 denotes a higher similarity between the query and passage in contrast to a score of 1.Hope it helps!
Kind Regards, Nandan Thakur
Thank you for your answer! That helps a lot!
Hello,
I see that you processed the raw MS MARCO dataset to include it in your unified BEIR-style dataset. After downloading and checking it, I have several questions.
test.tsv
come from? Given that the test set of MS MARCO passage retrieval task is not publicly available.test.tsv
stand for? I see there is only score 1 intrain.tsv
anddev.tsv
, while there are 0, 1, 2, 3 intest.tsv
.