Would be nice to support directly loading TREC run and qrel files directly

[cmacdonald]

or additionally, dataframe of the same format.

[cvangysel]

Do you mean like the following?

• https://github.com/cvangysel/pytrec_eval/blob/master/py/__init__.py#L16
• https://github.com/cvangysel/pytrec_eval/blob/master/py/__init__.py#L28
• https://github.com/cvangysel/pytrec_eval/blob/master/examples/trec_eval.py#L10

[cmacdonald]

Yes, I realised that after. Documenting these somewhere would help. Dataframe API would be good though :-)

[plopezgarcia]

Hi everyone. I think I might have already done part of the job.

This year I participated in the Precision Medicine track I wrote some small utilities on top of pytrec_eval that use dataframes as the main data structure and deal with little issues like:

• Loading a qrels file as a dataframe
• Convert that dataframe into the qrels dict format of pytrec_eval
• Dump a dataframe containing a run into trec_eval format

You can find a full example of usage here here.

Some things are still dependent on my code, but I think most of it could be generalized and reused? Let me know if I can help.