search

nyu-dl / dl4marco-bert

BSD 3-Clause "New" or "Revised" License
476 stars 87 forks source link

Difference between evaluation done using run treccar.py file and using the Anserini Script #52

Closed SinghShreyas closed 2 years ago

SinghShreyas commented 2 years ago

@rodrigonogueira4

Dear Rodrigo,

Fisrt of all, thanks for providing the code. It is of great help.

I just had one confusion. What is difference between the model evaluation done using run_treccar.py file and with anserini evaluation script. Bit confused about this. Please help me in this regard.

Thanks!

Best Regards, Shreyas

rodrigonogueira4 commented 2 years ago

Hi Shreyas, the evaluation with run_treccar.py outputs only a few metrics but it has the convenience that we can train and evaluate the model using the same script. Anserini uses trec_eval, which provides more metrics, but it needs first to generate a "run" file, so it is less convenient.

SinghShreyas commented 2 years ago

Hi Rodrigo,

Thanks a lot for your reply. That cleared my confusion. Really appreciate it.