Question about the result on "MSMARCO Passage Leadboard".

luyug / COIL

NAACL2021 - COIL Contextualized Lexical Retriever

Apache License 2.0

142 stars 27 forks source link

Question about the result on "MSMARCO Passage Leadboard". #11

Open hangzhang-nlp opened 2 years ago

hangzhang-nlp commented 2 years ago

I notice that C-Coil is at the top of the "MS MARCO Passage Ranking Leaderboard", the results are "0.427 on eval" and "0.443 on dev". But the result in https://github.com/luyug/COIL/tree/main/examples/c-coil is "0.3734 on MARCO DEV". I wonder why the gap is so big? Is it because of ensemble? I didn't find any relevant explanation in the Coil paper.