statisticianinstilettos / recmetrics

A library of metrics for evaluating recommender systems
MIT License
561 stars 99 forks source link

mapk shouldn't require actual and predicted have the same length #77

Closed mvonpohle closed 1 month ago

mvonpohle commented 9 months ago

This assertion check is incorrect. The actual parameter as used in _apk is expecting a list of true items and the predicted parameter is expecting a list of predicted items that can be true or false. See an example below where only A-C are true items and the prediction can be longer than the true list because it can contain false items. https://github.com/statisticianinstilettos/recmetrics/blob/b21222d3896f94960058824d1507414b98bf80f3/recmetrics/metrics.py#L236-L237

true_items = ["A","B","C"]
prediction = ["A","Z","B","X"]
metrics.mapk(actual=true_items, predicted=prediction, k = 3)
kenho211 commented 1 month ago

Looks like you should be using _apk? For average precision, they can be of different length so there is no assertion https://github.com/statisticianinstilettos/recmetrics/blob/b21222d3896f94960058824d1507414b98bf80f3/recmetrics/metrics.py#L163

The mapk is expecting both ground truth and predictions to be list of list. https://github.com/statisticianinstilettos/recmetrics/blob/b21222d3896f94960058824d1507414b98bf80f3/recmetrics/metrics.py#L220

mvonpohle commented 1 month ago

Ahh, yes. You're right, my bad.