search

usnistgov / trec_eval

Evaluation software used in the Text Retrieval Conference
232 stars 49 forks source link

Inconsistent arg naming: ndcg_cut_10 vs. ndcg_cut.10 #21

Closed lintool closed 4 years ago

lintool commented 4 years ago

trec_eval reports ndcg_cut_10 but the cmd-line option is -m ndcg_cut.10 I believe the same applies to other families of metrics also...

https://twitter.com/lintool/status/1232311794116501504

isoboroff commented 4 years ago

I'm pretty sure this is because, for the command line argument, the measure name is 'ndcg_cut', 10 is the cutoff which is an argument to the measure, and '.' is what separates the arg from the measure.

Let me check the source and see also how the so-called relational output format showed in since back when I added nDCG in the first place.

isoboroff commented 4 years ago

Confirmed. You can actually give all the cutoffs you want on the command line: -m ndcg_cut.10,20,30,50,100.

Closing.