msmarco-bot
commented
2 years ago Hi there, thanks for your submission!
I've gone ahead and run the evaluation... here are your results:
# Processing submission 20220419-futherMLM
Private key found!
Unpacking submission 20220419-futherMLM
Decrypting key...
Decrypting metadata...
Decrypting submission tarball...
Unpacking tarball...
Done!
Submission directory ./submissions/20220419-futherMLM
Verified: submission directory exists!
Verified: expected files appear in the submission directory!
Sanity checking run ./submissions/20220419-futherMLM/dev.txt.bz2
Run has 6974598 lines, 6980 unique queries.
Warning, 6980000 lines expected (1000 hits per query), instead 6974598 lines found!
Sanity checking run ./submissions/20220419-futherMLM/eval.txt.bz2
Run has 6832196 lines, 6837 unique queries.
Warning, 6837000 lines expected (1000 hits per query), instead 6832196 lines found!
Proceeding to evaluate:
Dev run MRR@10: 0.35666228907991104
Eval run MRR@10: 0.34699917117644113
Hi there, thanks for your submission!
Unfortunately, your submission currently does not conform to our submission guidelines. The team name "T403 in SCNU" is not sufficiently descriptive - see additional details at: https://github.com/microsoft/MSMARCO-Passage-Ranking-Submissions/#metadata-updates
Please include a clearly identifiable group/organization or the names of the individuals responsible for the submission; ideally, both. For example, the query "Tlab in SCNU" does not return any clearly identifiable results.
Can you please fix?