RecSys measures

[seanmacavaney]

@asash suggested incorporating RecSys measures into this package. I'm generally supportive and I think it'd be doable, but it'll take a bit of design/thought upfront.

We can use this issue to sketch it out.

Terminology:

• like use "user_id" instead of "query_id" and "item_id" instead of "doc_id"
• Does it automatically map between these conventions?

Measures:

• What new measures need to be supported?

Providers:

• What are the standard implementations? How easy will they be to integrate?
• We should be mindful of the dependencies introduced (aligned with the larger goal of reducing the number "required" dependencies)

Anything else?

[asash]

What new measures need to be supported? In the beginning, we can start with those metrics that are already there. The most popular RecSys metrics are nDCG@K and Recall@K (sometimes Recall@K is named "HIT@K").

Others that aren't there yet include RMSE, MSE, and AUC, but those could be added later. In any case, it shouldn't be too hard to implement those or use versions from scikit-learn.

Finally, there are beyond-accuracy metrics more specific to Recsys (though IR can also benefit from them). Some of those can include intra-list distance for diversity measurement, Catalogue Coverage, and some versions of popularity bias.

but I think allowing the use of RecSys-specific terminology would be a good start.