rating prediction accuracy comparision

[alanyuchenhou]

Here is a recent paper on rating prediction which we can compare with: Efficient recommendation methods using category experts for a large dataset. The specific experiment in the paper I will do has the follwoing specs:

• dataset: MovieLens 100k
• test set portion: 20%
• metric: MAE(mean absolute error)
• best result: MAE = 0.740

[alanyuchenhou]

OK, it made it. After more tweaking, our model can now consistently beat theirs: the testing MAE are in range [0.69, 0.72], depending on the randomness in the split of test, validate and train data from the entire dataset. An example fit process is shown below:

[alanyuchenhou]

Here is another paper we can compare with: A multi-level collaborative filtering method that improves recommendations . It's lowest MAE on MovieLen 1M dataset is ~0.79. Our estimator can get an MAE of 0.655. Below is the learning process:

[alanyuchenhou]

A multi-level collaborative filtering method that improves recommendations has experiments on both MovieLens 100K and MovieLens 1M. So I think this paper is all we need as a reference for prediction accuracy. We don't need Efficient recommendation methods using category experts for a large dataset any more.

[ghost]

We should compare against whichever approach has better performance.

[alanyuchenhou]

OK.

[alanyuchenhou]

This is the last issue in movie rating prediction, for which we have achieved good results. I'll close it now.