missing pytrec_eval measures

[cmacdonald]

This list is not exhaustive, but just the other measures more likely to be used.

• [x] iprec_at_recall
• [x] num_ret
• [x] num_rel_ret
• [x] num_q
• [x] num_rel
• [x] success
• [x] set_P
• [x] infAP

[cmacdonald]

The following nicknames might be handy:

• [x] 'official'
• [x] 'set'
• [x] 'all_trec'
• [x] 'iprec_at_recall'

[seanmacavaney]

For a pytrec_eval (nick)name -> irm mapping function, would something like this meet your needs?

ir_measures.convert_trec_name(name: str) -> List[BaseMeasure]

# Examples
ir_measures.convert_trec_name('map') -> [AP]
ir_measures.convert_trec_name('P_5') -> [P@5]
ir_measures.convert_trec_name('P_5,10') -> [P@5, P@10]
ir_measures.convert_trec_name('P.5') -> [P@5]
ir_measures.convert_trec_name('P.5,10') -> [P@5, P@10]
ir_measures.convert_trec_name('P') -> [P@5, P@10, P@15, P@20, P@30, P@100, P@200, P@500, P@1000] # expando
ir_measures.convert_trec_name('ndcg') -> [nDCG]
ir_measures.convert_trec_name('ndcg_cut') -> [nDCG@5, nDCG@10, ...] # expando
ir_measures.convert_trec_name('official') -> [NumRet, ...] # all those from the family

[cmacdonald]

Also:

ir_measures.convert_trec_name('ndcg_cut_5') -> [nDCG@5]

Your implementation might come from: https://github.com/terrier-org/pyterrier/blob/ir_measures/pyterrier/pipelines.py#L64

[seanmacavaney]

Remaining measures (identified via supported_measures in pytrec_eval):

• [ ] Rndcg
• [ ] num_nonrel_judged_ret
• [ ] 11pt_avg
• [ ] relative_P
• [ ] gm_map
• [ ] binG
• [ ] ndcg_rel
• [ ] Rprec_mult
• [ ] gm_bpref
• [ ] G
• [ ] utility
• [x] set_recall
• [x] set_F
• [x] set_relative_P
• [x] set_map
• [ ] runid
• [ ] relstring