Different number of queries in collectionandqueries.tar.gz and top1000.dev.tar.gz

spacemanidol / MSMARCO

Utilities, Baselines, Statistics and Descriptions Related to the MSMARCO DATASET

MIT License

189 stars 41 forks source link

Different number of queries in collectionandqueries.tar.gz and top1000.dev.tar.gz #29

Closed rodrigonogueira4 closed 5 years ago

rodrigonogueira4 commented 5 years ago

There are different number of unique queries in dev set in collectionandqueries.tar.gz (101093) and top1000.dev.tar.gz (6980).

Could you please update collectionandqueries.tar.gz with the smaller version? Otherwise, 101k queries take too long to evaluate.

Same problem in the eval set.

spacemanidol commented 5 years ago

Ive gone ahead an updated the collectionsandqueries.tar.gz to include queries and queries..small.tsv. Queries includes all queries in the split and queries.small includes only the ones referenced in the current public top1000..