search

tira-io / ir-experiment-platform

28 stars 3 forks source link

TIRA Evaluator: Extract top-docs for DiffIR and render them by default. #5

Closed mam10eks closed 1 year ago

mam10eks commented 1 year ago

At the moment, one has to execute multiple commands to render the serps. The goal is to do this by default in the evaluation, so that running the evaluation directly creates everything so one only has to execute a single command.

mam10eks commented 1 year ago

This is now implemented, there are only minor issues for wapo, argsme, and nfcorpus for which are serialization issues but this should be easy to fix.

mam10eks commented 1 year ago

This is now implemented and documented.