AveragePrecision in recsys.evaluation.ranking

[lisiqi]

Hi Oscar, The calculation of AveragePrecision in recsys.evaluation.ranking is not correct. The returned value should be sum(p_at_k)/number of relevant items, rather than sum(p_at_k)/hits.

In your document --> evaluation, the corresponding part also needs to be changed.

from recsys.evaluation.ranking import AveragePrecision

ap = AveragePrecision()

GT = [1,2,3,4,5] q = [1,3,5] ap.load(GT, q) ap.compute() # returns 1.0, should return 0.6

GT = [1,2,3,4,5] q = [99,3,5] ap.load(GT, q) ap.compute() # returns 0.5833335, should return 0.23333

Kind Regards, Siqi

[ocelma]

Hi Siqi,

Thanks! I need to look into the code. It's been a while...

I think instead of (line 148):

return sum(p_at_k)/hits

I should do something like:

return sum(p_at_k)/len(self._ground_truth)