Closed cthoyt closed 2 years ago
Merging #23 (5a4b281) into main (6d166ff) will increase coverage by
0.00%
. The diff coverage is100.00%
.
@@ Coverage Diff @@
## main #23 +/- ##
=======================================
Coverage 99.86% 99.87%
=======================================
Files 15 15
Lines 754 787 +33
=======================================
+ Hits 753 786 +33
Misses 1 1
Impacted Files | Coverage Δ | |
---|---|---|
rexmex/metrics/ranking.py | 100.00% <100.00%> (ø) |
|
tests/unit/test_metrics.py | 100.00% <100.00%> (ø) |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update 6d166ff...5a4b281. Read the comment docs.
Looks great @cthoyt! Thank you - we will add you to the bib.
Summary
This PR follows the style of the implementation of the mean reciprocal rank to add an implementation of the rank helper function, the mean rank, and the geometric mean rank (originally described here).
Changes
rank()
helper function similar toreciprocal_rank()
helper functionmean_rank()
function, corresponding to the commonly used mean rank (MR) metric in KGEM evaluationgmean_rank()
function, corresponding to recently introduced geometric mean rank (GMR) metric useful for KGEM evaluationQuestions
reciprocal_rank()
function simply withreturn 1.0 / rank(relevant_item, recommendation)
. I'm not sure if there was a specific reason why MRR was implemented but not MR, but the logic is heavily related so it might even be more instructive to have one implemented in terms of the other