search

AmenRa / ranx

⚡️A Blazing-Fast Python Library for Ranking Evaluation, Comparison, and Fusion 🐍
https://amenra.github.io/ranx
MIT License
427 stars 23 forks source link

[BUG] Missing results causes AssertionError #39

Closed amallia closed 1 year ago

amallia commented 1 year ago

Missing results causes AssertionError: Qrels and Run query ids do not match You should make check_keys optional in evaluate because sometimes queries do not return any results for lexical-based systems.

https://github.com/AmenRa/ranx/blob/da0aa5246513acb87ec7ac7f516194fea464a2c1/ranx/meta/evaluate.py#L128

AmenRa commented 1 year ago

Hi, I've added the make_comparable function to Run in v0.3.7.

run.make_comparable(qrels)

The run will be modified based on the qrels passed as an argument. Query results not appearing in qrels will be removed from run to avoid unpredictable behavior or other bugs. For convenience, I also added the boolean parameter make_comparable to evaluate and compare (it defaults to False).

Please, consider giving ranx a star if you like it.

Best,

Elias