Submission 20211116-sembly

[ghost]

All CLA requirements met.

[msmarco-bot]

Hi there, thanks for your submission!

I've gone ahead and run the evaluation... here are your results:

# Processing submission 20211116-sembly

Private key found!
Unpacking submission 20211116-sembly
Decrypting key...
Decrypting metadata...
Decrypting submission tarball...
Unpacking tarball...
Done!

Submission directory ./submissions/20211116-sembly
Verified: submission directory exists!
Verified: expected files appear in the submission directory!

Sanity checking run ./submissions/20211116-sembly/dev.txt.bz2
Run has 6668967 lines, 6980 unique queries.
Warning, 6980000 lines expected (1000 hits per query), instead 6668967 lines found!

Sanity checking run ./submissions/20211116-sembly/eval.txt.bz2
Run has 6515736 lines, 6837 unique queries.
Warning, 6837000 lines expected (1000 hits per query), instead 6515736 lines found!

Proceeding to evaluate:

Dev run MRR@10: 0.3375986946832207
Eval run MRR@10: 0.33309838391014324