search

hltcoe / patapsco

Cross language information retrieval pipeline
Other
18 stars 6 forks source link

Integration with ir-measures #35

Open eugene-yang opened 2 years ago

eugene-yang commented 2 years ago

Since NeuCLIR track has tentatively made the decision to use evaluation metrics beyond the support of trec_eval, e.g. Rank-Biased Precision, it would be ideal to support them in Patapsco.

As trec_eval is kind of frozen at this point, supporting them through ir-measures would be more ideal (#27).

eugene-yang commented 2 years ago

@cash Rank-Biased Precision will be the one we need to test with. https://ir-measur.es/en/latest/measures.html#nrbp