search

usnistgov / trec_eval

Evaluation software used in the Text Retrieval Conference
232 stars 49 forks source link

trec_eval_compressed #2

Closed cmacdonald closed 8 years ago

cmacdonald commented 8 years ago

Allows use of trec_eval in pipes, or with compressed qrels or res files.

isoboroff commented 8 years ago

Have to mull this one. I'm not sure I want to start packaging wrappers. I don't think reading the entire file into memory is a good idea. If the decompression subprocesses fail the script can write partial files to the temporary outputs.

cmacdonald commented 8 years ago

I'm not sure I want to start packaging wrappers

Of course, your choice. This is a script we've been using for years. I was wondering about Windows compatibility, but the trec_eval Makefile definitively wont port either.

I don't think reading the entire file into memory is a good idea

Fair. I can fix that in the Perl script trivially. A related point is that trec_eval already operates by loading entire files into memory (which it achieves IIRC by seeking to the end of the file, and then back to the start), and hence isn't able to operate on a pipe. The advantage using this script is that evaluation based on an input pipe is supported transparently.

If the decompression subprocesses fail the script can write partial files to the temporary outputs.

$? can be inspected to determine subprocess failure.

cmacdonald commented 8 years ago

Ian: updated pull request addressing error checking and exit codes.

I tested to verify the script failed if gzip didnt exist, or exited with non-zero exit code.

isoboroff commented 8 years ago

I created a new branch version-10.0-dev. Could you please re-target your pull request to that branch, and I'll pull it in there.

cmacdonald commented 8 years ago

See #4