read_results optimization

terrier-org / pyterrier

A Python framework for performing information retrieval experiments, building on http://terrier.org/

https://pyterrier.readthedocs.io/

Mozilla Public License 2.0

397 stars 63 forks source link

read_results optimization #421

Closed seanmacavaney closed 6 months ago

seanmacavaney commented 6 months ago

Per my testing, this speeds up read_results by around 50% for a reasonably large result file (msmarco-passage/dev/small results with 1000 per query).

cmacdonald commented 6 months ago

lgtm! thanks