AmenRa / ranx

⚡️A Blazing-Fast Python Library for Ranking Evaluation, Comparison, and Fusion 🐍
https://amenra.github.io/ranx
MIT License
427 stars 23 forks source link

[BUG] dcg and dcg_burges do not work in the compare function #42

Closed mpetri closed 1 year ago

mpetri commented 1 year ago

Describe the bug

when adding the newly available dcg or dcg_burges metric in the compare function I get this error:

report = compare(
    qrels=qrels,
    runs=runs,
    metrics=["recall@10","ndcg","rbp.90","rbp.50","dcg_burges"],
    max_p=0.05,   # P-value threshold
    stat_test='student'
)
Traceback (most recent call last):
  File "/local/home/mkp/data/gap2kic/eval/./run.py", line 32, in <module>
    print(report)
  File "/home/mkp/.asdf/installs/python/3.10.9/lib/python3.10/site-packages/ranx/data_structures/report.py", line 338, in __str__
    return self.to_table()
  File "/home/mkp/.asdf/installs/python/3.10.9/lib/python3.10/site-packages/ranx/data_structures/report.py", line 143, in to_table
    label = self.get_metric_label(x)
  File "/home/mkp/.asdf/installs/python/3.10.9/lib/python3.10/site-packages/ranx/data_structures/report.py", line 122, in get_metric_label
    return f"{metric_labels[m]}"
KeyError: 'dcg_burges'

however in the same file when I do

res = evaluate(qrels, run, ["recall@10","ndcg","rbp.90","rbp.50","dcg","dcg_burges"])

everything works as intended.

mpetri commented 1 year ago

I think metric labels are missing here: https://github.com/AmenRa/ranx/blob/cf7b00c829a223c988c080ab39562637527346ca/ranx/data_structures/report.py#L24C38-L24C38

AmenRa commented 1 year ago

Yep, I forget about them. Thanks for pointing it out! Fixed in v0.3.14