search

sebastian-hofstaetter / neural-ranking-drmm

Implementation and Evaluation of "A Deep Relevance Matching Model for Ad-hoc Retrieval"
Apache License 2.0
36 stars 8 forks source link

Have you reproduced the result? #1

Closed guotong1988 closed 5 years ago

guotong1988 commented 5 years ago

Result in paper. Thank you! @sebastian-hofstaetter

sebastian-hofstaetter commented 5 years ago

Hi, No, unfortunately I was not able to reproduce DRMM results for Robust 04 - in my current research on neural IR models using MS MARCO I also observed DRMM is not outperforming BM25 (In contrast other models do outperform BM25 by a large margin on the MS MARCO passage re-ranking task). If you want to know more about the better models visit (with re-implementations in PyTorch): https://github.com/sebastian-hofstaetter/sigir19-neural-ir

guotong1988 commented 5 years ago

Thank you!