Open seanmacavaney opened 9 months ago
What new measures need to be supported? In the beginning, we can start with those metrics that are already there. The most popular RecSys metrics are nDCG@K and Recall@K (sometimes Recall@K is named "HIT@K").
Others that aren't there yet include RMSE, MSE, and AUC, but those could be added later. In any case, it shouldn't be too hard to implement those or use versions from scikit-learn.
Finally, there are beyond-accuracy metrics more specific to Recsys (though IR can also benefit from them). Some of those can include intra-list distance for diversity measurement, Catalogue Coverage, and some versions of popularity bias.
but I think allowing the use of RecSys-specific terminology would be a good start.
@asash suggested incorporating RecSys measures into this package. I'm generally supportive and I think it'd be doable, but it'll take a bit of design/thought upfront.
We can use this issue to sketch it out.
Terminology:
Measures:
Providers:
Anything else?