search

joaopalotti / trectools

A simple toolkit to process TREC files in Python.
https://pypi.python.org/pypi/trectools
BSD 3-Clause "New" or "Revised" License
163 stars 32 forks source link

Malformed lines #34

Open lironT74 opened 2 years ago

lironT74 commented 2 years ago

Hi @joaopalotti, It seems that for malformed lines, such as: image trec_eval throws: trec_eval.get_results: Malformed line 790 While trec_tools does not.

I am not sure that this is a bad thing but perhaps a warning will suite here? Unfortunately I am a bit swamped lately so I am not available to offer a fix myself. Thank you!

joaopalotti commented 2 years ago

Hi @lironT74, thank you very much for identifying this mismatch between trec_eval and trectools. While ago, I created a set of validation scripts for CLEF eHealth. Their goal was to verify this type of error and other similar problems such as non-sequential document rank or non-decreasing score for a given topic.

Having this kind of check integrated into trectools would be amazing!

I will leave this issue with a help-wanted flag and hope to get help from the amazing IR community!