Submission 20220112-revl147

microsoft / MSMARCO-Passage-Ranking-Submissions

Submission archive for the MS MARCO passage ranking leaderboard

https://microsoft.github.io/MSMARCO-Passage-Ranking-Submissions/leaderboard

MIT License

12 stars 30 forks source link

Submission 20220112-revl147 #29

Closed RevL147 closed 2 years ago

ghost commented 2 years ago

All CLA requirements met.

msmarco-bot commented 2 years ago

Hi there, thanks for your submission!

Unfortunately, your submission currently does not conform to our submission guidelines. The team name "Anonymous" and model description "Anonymous" are not allowed, even for a model under embargo (which we will respect).

See additional details at: https://github.com/microsoft/MSMARCO-Passage-Ranking-Submissions/#metadata-updates

Can you please fix?

RevL147 commented 2 years ago

Hi there, thanks for your submission!

Unfortunately, your submission currently does not conform to our submission guidelines. The team name "Anonymous" and model description "Anonymous" are not allowed, even for a model under embargo (which we will respect).

See additional details at: https://github.com/microsoft/MSMARCO-Passage-Ranking-Submissions/#metadata-updates

Can you please fix?

fixed

RevL147 commented 2 years ago

Already updated for all three files.

msmarco-bot commented 2 years ago

Unfortunately, your submission still does not conform to our submission guidelines. The team name is not sufficiently descriptive - see additional details at: https://github.com/microsoft/MSMARCO-Passage-Ranking-Submissions/#metadata-updates

Please include a clearly identifiable group/organization or the names of the individuals responsible for the submission. For example, if I query your name with a search engine, I should get clearly identifiable results.

Can you please fix?

RevL147 commented 2 years ago

Unfortunately, your submission still does not conform to our submission guidelines. The team name is not sufficiently descriptive - see additional details at: https://github.com/microsoft/MSMARCO-Passage-Ranking-Submissions/#metadata-updates

Please include a clearly identifiable group/organization or the names of the individuals responsible for the submission. For example, if I query your name with a search engine, I should get clearly identifiable results.

Can you please fix?

Hi, I have changed team name, please check again, thanks!

msmarco-bot commented 2 years ago

Hi there, thanks for your submission!

I've gone ahead and run the evaluation... here are your results:

# Processing submission 20220112-revl147

Private key found!
Unpacking submission 20220112-revl147
Decrypting key...
Decrypting metadata...
Decrypting submission tarball...
Unpacking tarball...
Done!

Submission directory ./submissions/20220112-revl147
Verified: submission directory exists!
Verified: expected files appear in the submission directory!

Sanity checking run ./submissions/20220112-revl147/dev.txt.bz2
Run has 6980000 lines, 6980 unique queries.

Sanity checking run ./submissions/20220112-revl147/eval.txt.bz2
Run has 6837000 lines, 6837 unique queries.

Proceeding to evaluate:

Dev run MRR@10: 0.4504280370218775
Eval run MRR@10: 0.4353413151131454