Submission 20211013-ZBERT

[zyznull]

Submission 20211011-rom

[ghost]

All CLA requirements met.

[msmarco-bot]

Hi there, thanks for your submission!

Unfortunately, your submission currently does not conform to our submission guidelines. An "anonymous" team name is not allowed - see additional details at: https://github.com/microsoft/MSMARCO-Document-Ranking-Submissions/#metadata-updates

(Sorry, we have yet to update the submission instructions in this repo, but our guidelines are the same across doc and passage submissions).

Can you please fix?

[msmarco-bot]

Your submission still does not conform to our submission guidelines:

It is generally expected that the team description in the metadata file will include the name of the organization (e.g., university or company). In many cases, submissions explicitly list the contributors of the run.

"Yanzhao" is not sufficiently informative. If this is a personal submission, please include your full name. If you are part of an organization (university or company), we expect the name of the organization to be included in the team.

[msmarco-bot]

Hi there, thanks for your submission!

I've gone ahead and run the evaluation... here are your results:

# Processing submission 20211013-ZBERT

Private key found!
Unpacking submission 20211013-ZBERT
Decrypting key...
Decrypting metadata...
Decrypting submission tarball...
Unpacking tarball...
Done!

Submission directory ./submissions/20211013-ZBERT
Verified: submission directory exists!
Verified: expected files appear in the submission directory!

Sanity checking run ./submissions/20211013-ZBERT/dev.txt.bz2
Run has 698000 lines, 6980 unique queries.
Warning, 6980000 lines expected (1000 hits per query), instead 698000 lines found!

Sanity checking run ./submissions/20211013-ZBERT/eval.txt.bz2
Run has 683700 lines, 6837 unique queries.
Warning, 6837000 lines expected (1000 hits per query), instead 683700 lines found!

Proceeding to evaluate:

Dev run MRR@10: 0.44110224223404837
Eval run MRR@10: 0.427715964256115