Closed f766ece33d1fd43bfa618eb779e9ea5b closed 1 year ago
Hi there, thanks for your submission!
I've gone ahead and run the evaluation... here are your results:
# Processing submission 20220929-doc2query--electra Private key found! Unpacking submission 20220929-doc2query--electra Decrypting key... Decrypting metadata... Decrypting submission tarball... Unpacking tarball... Done! Submission directory ./submissions/20220929-doc2query--electra Verified: submission directory exists! Verified: expected files appear in the submission directory! Sanity checking run ./submissions/20220929-doc2query--electra/dev.txt.bz2 Run has 6975912 lines, 6980 unique queries. Warning, 6980000 lines expected (1000 hits per query), instead 6975912 lines found! Sanity checking run ./submissions/20220929-doc2query--electra/eval.txt.bz2 Run has 6833644 lines, 6837 unique queries. Warning, 6837000 lines expected (1000 hits per query), instead 6833644 lines found! Proceeding to evaluate: Dev run MRR@10: 0.3226108041115198 Eval run MRR@10: 0.324690758269081
Hi there, thanks for your submission!
I've gone ahead and run the evaluation... here are your results: