Closed amallia closed 1 year ago
Hi, I've added the make_comparable
function to Run
in v0.3.7.
run.make_comparable(qrels)
The run
will be modified based on the qrels
passed as an argument.
Query results not appearing in qrels
will be removed from run
to avoid unpredictable behavior or other bugs.
For convenience, I also added the boolean parameter make_comparable
to evaluate and compare (it defaults to False
).
Please, consider giving ranx
a star if you like it.
Best,
Elias
Missing results causes AssertionError: Qrels and Run query ids do not match You should make
check_keys
optional in evaluate because sometimes queries do not return any results for lexical-based systems.https://github.com/AmenRa/ranx/blob/da0aa5246513acb87ec7ac7f516194fea464a2c1/ranx/meta/evaluate.py#L128