AttributeError: module 'surprise.dataset' has no attribute 'load_from_df'

[Mayank-Bhatia]

Hello,

I have updated dataset.py in my library, so it does include load_from_df(), however when I try to call it, I get the above attribute error.

My data set is the standard three column data containing user id, item id, rating.

Any advise would be appreciated.

Thanks, Mayank

[NicolasHug]

Hi,

The method is accessed with surprise.Dataset.load_from_df, with an upper-case 'D'.

Nicolas

[Mayank-Bhatia]

Thank you for the swift response. This allowed me to call the function.

I am now having a follow-up error I am not sure how to tackle. For context, I am loading a pandas df through the load_from_df function: reader = Reader(line_format='user item rating', rating_scale=(1, 10)) data = Dataset.load_from_df(anime_rating[['user_id', 'anime_id', 'rating']], reader)

But when in evaluating KNNWithMeans's performance: sim_options = {'name': 'pearson', 'user_based': False} algo = KNNWithMeans(k=20, min_k=11, sim_options=sim_options) perf = evaluate(algo, data, measures=['RMSE','MAE'])

I get this error: AttributeError: 'int' object has no attribute 'sqrt'

Sorry to bother you again!

[NicolasHug]

Hi,

Please give me your whole code and the details of the error message, because here I cannot know where the error is happening

Nicolas

[Mayank-Bhatia]

Here's the code:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from surprise import KNNWithMeans
from surprise import Dataset
from surprise import evaluate, print_perf
from surprise import Reader

anime_rating = pd.read_csv('rating.csv') # the data
anime_rating = anime_rating[anime_rating.rating > 0] # keep ratings between 1-10
anime_rating['user_anime_count'] = anime_rating.groupby('user_id')['user_id'].transform(np.count_nonzero)
anime_rating = anime_rating[anime_rating.user_anime_count > 19] # keep users that have seen at least 20 anime
anime_rating = anime_rating.drop('user_anime_count', axis=1) # having served its purpose, drop the user count column

#loading from pandas dataframe
reader = Reader(line_format='user item rating', rating_scale=(1, 10))
data = Dataset.load_from_df(anime_rating[['user_id', 'anime_id', 'rating']], reader)

#testing the knnwithmeans
sim_options = {'name': 'pearson', # compute Pearson correlation coefficient between all pairs of items
               'user_based': False  # compute similarities between items
               }
algo = KNNWithMeans(k=20, min_k=11, sim_options=sim_options)
perf = evaluate(algo, data, measures=['RMSE','MAE'])

And here's the error:

Evaluating RMSE, MAE of algorithm KNNWithMeans.

------------
Fold 1
Computing the pearson similarity matrix...
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-14-017eb59430bf> in <module>()
      3                }
      4 algo = KNNWithMeans(k=20, min_k=11, sim_options=sim_options)
----> 5 perf = evaluate(algo, data, measures=['RMSE','MAE'])

c:\users\mike\appdata\local\programs\python\python35\lib\site-packages\surprise\evaluate.py in evaluate(algo, data, measures, with_dump, dump_dir, verbose)
     63 
     64         # train and test algorithm. Keep all rating predictions in a list
---> 65         algo.train(trainset)
     66         predictions = algo.test(testset, verbose=(verbose == 2))
     67 

c:\users\mike\appdata\local\programs\python\python35\lib\site-packages\surprise\prediction_algorithms\knns.py in train(self, trainset)
    166 
    167         SymmetricAlgo.train(self, trainset)
--> 168         self.sim = self.compute_similarities()
    169 
    170         self.means = np.zeros(self.n_x)

c:\users\mike\appdata\local\programs\python\python35\lib\site-packages\surprise\prediction_algorithms\algo_base.py in compute_similarities(self)
    229         try:
    230             print('Computing the {0} similarity matrix...'.format(name))
--> 231             sim = construction_func[name](*args)
    232             print('Done computing similarity matrix.')
    233             return sim

c:\users\mike\appdata\local\programs\python\python35\lib\site-packages\surprise\similarities.pyx in surprise.similarities.pearson (surprise/similarities.c:6558)()

AttributeError: 'int' object has no attribute 'sqrt'

[Mayank-Bhatia]

If it helps, here's the first 10 columns of the dataframe before using load_from_df:

user_id anime_id rating 156 3 20 8 157 3 154 6 158 3 170 9 159 3 199 10 160 3 225 9 161 3 341 6 162 3 430 7 163 3 527 7 164 3 552 7 165 3 813 10

[NicolasHug]

There are four columns in your example, is this normal?

Anyway the issue looks very weird and I don't see an obvious solution. I'll come back to it tomorrow (it's almost 11pm here).

As a side note, you can see that I formatted your previous post: please consider doing it next time, it really helps reading ;)

Nicolas

[Mayank-Bhatia]

I think the first column is the index, and anime_rating.shape returns (6164987, 3)

Sorry for not formatting earlier.

Mayank

[Mayank-Bhatia]

Actually, just ran the evaluation with msd instead of pearson, and discovered no issue. Cosine returned the following while computing the similarity matrix: ...\surprise\prediction_algorithms\algo_base.py:231: RuntimeWarning: invalid value encountered in sqrt sim = construction_func[name](*args)

[NicolasHug]

Looks like you always have a problem with np.sqrt... Really special.

Can you try a new clean installation and see how it goes?

• install numpy
• install cython
• then pip install https://github.com/NicolasHug/Surprise/archive/master.zip

[NicolasHug]

I cannot reproduce the issue on my side. It works well with MSD, Pearson and Cosine.

Here is my code (I used the Kaggle dataset as you did, right?)

import surprise
import pandas as pd

df = pd.DataFrame.from_csv('rating.csv')
df.reset_index(level=0, inplace=True)  # so that columns are user_id, anime_id and rating
df = df[df.rating>0]

# I'm only loading the first 1000 rows to save time
reader = surprise.dataset.Reader(rating_scale=(1, 10))
data = surprise.Dataset.load_from_df(df.head(1000)[['user_id', 'anime_id', 'rating']], reader)

sim_options = {'name': 'cosine', 'user_based': False}
algo = surprise.KNNWithMeans(k=20, min_k=11, sim_options=sim_options)
surprise.evaluate(algo, data, measures=['RMSE', 'MAE'])

Can you try that (on a clean environment, see previous post) and see how it goes?

Nicolas

[Mayank-Bhatia]

I actually ran it in a clean environment last night, and it all worked out.

Thank you :)

[NicolasHug]

Cool! Very weird problem ^^