search

caserec / CaseRecommender

Case Recommender: A Flexible and Extensible Python Framework for Recommender Systems
MIT License
482 stars 91 forks source link

Overflow error encountered in Matrixfactorization and SVDPlusPlus #38

Closed srijang97 closed 4 years ago

srijang97 commented 4 years ago

Hi. I am trying out different recommendation algorithms with random hyperparameters. Both MatrixFactorization and SVDPlusPlus produce the following errors:

C:\Users\guptasr\AppData\Local\Continuum\anaconda3\envs\fyp2\lib\site-packages\caserec\recommenders\rating_prediction\matrixfactorization.py:146: RuntimeWarning: overflow encountered in double_scalars
  error_final += (eui ** 2.0)

C:\Users\guptasr\AppData\Local\Continuum\anaconda3\envs\fyp2\lib\site-packages\caserec\recommenders\rating_prediction\matrixfactorization.py:153: RuntimeWarning: overflow encountered in multiply
  delta_u = np.subtract(np.multiply(eui, i_f), np.multiply(self.delta, u_f))

C:\Users\guptasr\AppData\Local\Continuum\anaconda3\envs\fyp2\lib\site-packages\caserec\recommenders\rating_prediction\matrixfactorization.py:154: RuntimeWarning: overflow encountered in multiply
  delta_i = np.subtract(np.multiply(eui, u_f), np.multiply(self.delta, i_f))

C:\Users\guptasr\AppData\Local\Continuum\anaconda3\envs\fyp2\lib\site-packages\caserec\recommenders\rating_prediction\matrixfactorization.py:157: RuntimeWarning: invalid value encountered in add
  self.p[user] += np.multiply(self.learn_rate, delta_u)

C:\Users\guptasr\AppData\Local\Continuum\anaconda3\envs\fyp2\lib\site-packages\caserec\recommenders\rating_prediction\matrixfactorization.py:158: RuntimeWarning: invalid value encountered in add
  self.q[item] += np.multiply(self.learn_rate, delta_i)

The dataset is as follows: train data:: 1890 users and 16551 items (83550 interactions) | sparsity:: 99.73% test data:: 1864 users and 3973 items (9284 interactions) | sparsity:: 99.87%

The input parameters to the MatrixFactorization class for this run were: 'mf_delta': 0.08706786244826163, 'mf_epochs': 19.0, 'mf_factors': 190.0, 'mf_learn_rate': 0.09982193107968043

By removing the learn_rate and delta input parameters, the error goes away!!

Also, using the same configuration on a different dataset (Movielens100K) did not produce any error. train data:: 610 users and 9003 items (80668 interactions) | sparsity:: 98.53% test data:: 610 users and 5089 items (20168 interactions) | sparsity:: 99.35%

Kindly advise on how to solve this issue!!

srijang97 commented 4 years ago

Issue resolved after changing the feedback range to [0, 5]