search

aliannejadi / ClariQ

ClariQ: SCAI Workshop data challenge on conversational search clarification.
130 stars 26 forks source link

How do you get <relevance_score> in train.qrel #7

Open ShiyuNee opened 1 year ago

ShiyuNee commented 1 year ago

I'd like to know that why the relevance score can be 0 and i notice that there are only 628 facets that have documents whose relevance score is larger than 0. However, there are 1070 facets in train.qrel.

ShiyuNee commented 1 year ago

There are 10 facets with no relevant documents in train.tsv. The facets are: F0322 F0155 F0140 F0349 F0041 F0753 F0365 F0276 F0166 F0294

aliannejadi commented 1 year ago

We have adapted the qrel files of the Web track where a relevance score of "0" means that the document was assessed but judged as irrelevant. The difference with other irrelevant documents is that they are not assessed. That's true, there are some facets with no relevant documents.