Evaluating precision@k/recall@k for users with <k "known positives"

lyst / lightfm

A Python implementation of LightFM, a hybrid recommendation algorithm.

Apache License 2.0

4.77k stars 691 forks source link

Open keryums opened 4 years ago

keryums commented 4 years ago

Hi,

I have a question about the precision@k and recall@k evaluation functions. I can see from the source code that precision is calculated as:

precision = np.squeeze(np.array(ranks.sum(axis=1))) / k

Wouldn't this calculation underestimate performance if I have a users who have voted/rated <k items?

luthfianto commented 3 years ago

@keryums I have the same problem too, most of my users only transact once. How is your current solution?