question about the `2019qrels-docs` file on Document ranking dataset

[Albert-Ma]

For the test set, why there are only 43 queries in the 2019qrels-docs ? I'm confused since it has 200 queries for the whole test set.

[bmitra-msft]

From the TREC 2019 Deep Learning track overview: "Participants were provided with an initial set of 200 test queries, then NIST later selected 43 queries during the pooling and judging process, based on budget constraints and with the goal of producing a reusable test collection. The same 200 queries were used for submissions in both tasks, while the selected 43 queries for each task were overlapping but not identical. The full judging process is described in Section 5." Source: https://arxiv.org/pdf/2003.07820.pdf