MSMARCO no matching qids found

Georgetown-IR-Lab / OpenNIR

An end-to-end neural ad-hoc ranking pipeline.

https://opennir.net

MIT License

148 stars 25 forks source link

MSMARCO no matching qids found #30

Open aminbigdeli opened 3 years ago

aminbigdeli commented 3 years ago

I tried to train conv_knrm on the MSMARCO dataset by running the following command: scripts/pipeline.sh config/conv_knrm config/msmarco However I faced "No matching QIDs found. Are you sure you are scoring the evaluation set?" error.

Would you please help me with that? It seems to me that there should be a problem with OpeNIR because it downloads the entire dataset itself.

seanmacavaney commented 3 years ago

From correspondence with @aminbigdeli -- it seems the minidev.qrels file was empty. I need to look into why this happened + correct the bug.