search

microsoft / MSMARCO-Passage-Ranking-Submissions

Submission archive for the MS MARCO passage ranking leaderboard
https://microsoft.github.io/MSMARCO-Passage-Ranking-Submissions/leaderboard
MIT License
12 stars 30 forks source link

Submission 20211012-informagus #8

Closed EmmaGerritse closed 3 years ago

ghost commented 3 years ago

CLA assistant check
All CLA requirements met.

msmarco-bot commented 3 years ago

Hi there, thanks for your submission!

Unfortunately, your submission currently does not conform to our submission guidelines. We ask that you clearly identify the team (e.g., the name of a university) - see additional details at: https://github.com/microsoft/MSMARCO-Document-Ranking-Submissions/#metadata-updates

(Sorry, we have yet to update the submission instructions in this repo, but our guidelines are the same across doc and passage submissions).

Can you please fix?

msmarco-bot commented 3 years ago

Hi there, thanks for your submission!

I've gone ahead and run the evaluation... here are your results:

# Processing submission 20211012-informagus

Private key found!
Unpacking submission 20211012-informagus
Decrypting key...
Decrypting metadata...
Decrypting submission tarball...
Unpacking tarball...
Done!

Submission directory ./submissions/20211012-informagus
Verified: submission directory exists!
Verified: expected files appear in the submission directory!

Sanity checking run ./submissions/20211012-informagus/dev.txt.bz2
Run has 6974598 lines, 6980 unique queries.
Warning, 6980000 lines expected (1000 hits per query), instead 6974598 lines found!

Sanity checking run ./submissions/20211012-informagus/eval.txt.bz2
Run has 6832196 lines, 6837 unique queries.
Warning, 6837000 lines expected (1000 hits per query), instead 6832196 lines found!

Proceeding to evaluate:

Dev run MRR@10: 0.38138115704734654
Eval run MRR@10: 0.37088159431292383