Closed seanmacavaney closed 2 years ago
Confirmed that the original scoreddocs are for the small set: https://github.com/microsoft/MSMARCO-Passage-Ranking#top1000
The same applies to msmarco-passage/eval
(should be moved to msmarco-passage/eval/small
).
Describe the bug
The scoreddocs of
msmarco-passage/dev
only appear to contain records for queries found inmsmarco-passage/dev/small
. Is there a more complete version of the scoreddocs for the full dev set, or do scoreddocs only apply to thesmall
version?Affected dataset(s)
msmarco-passage/dev
(shouldn't provide scoreddocs---unless larger set of scoreddocs are available somewhere)msmarco-passage/dev/judged
(shouldn't provide scoreddocs---unless larger set of scoreddocs are available somewhere)msmarco-passage/dev/small
(should provide the current scoreddocs)Expected behavior
If the scoreddocs only apply to
msmarco-passage/dev/small
, they should only be present for this subset. If there's a larger version available, they should be available instead viamsmarco-passage/dev
(andmsmarco-passage/dev/judged
).