TypeError: 'numpy.float64' object cannot be interpreted as an index

[SeekPoint]

ub16hp@UB16HP:~/ub16_prj/FMG$ gcc -fPIC --shared setVal.c -o setVal.so ub16hp@UB16HP:~/ub16_prj/FMG$ gcc -fPIC --shared partXY.c -o partXY.so ub16hp@UB16HP:~/ub16_prj/FMG$ python mf_features_generator.py yelp-50k all 1 data: data/yelp-50k/exp_split/1/, path_str: all finish load data from data/yelp-50k/exp_split/1/sim_res/path_count/UPBCatB_top500.res, cost 32.46 seconds, users: 13663, items=8133 start generate mf features, (K, eps, reg, iters) = (10, 10, 10, 500) Traceback (most recent call last): File "mf_features_generator.py", line 165, in run_all_yelp() File "mf_features_generator.py", line 91, in run_all_yelp run(path_str) File "mf_features_generator.py", line 49, in run U,V = mf.run() File "/home/ub16hp/ub16_prj/FMG/mf.py", line 122, in run X = cm((self.data[:,2], (self.data[:,0], self.data[:,1]))) #index starting from 0 File "/usr/local/lib/python2.7/dist-packages/scipy/sparse/compressed.py", line 51, in init other = self.class(coo_matrix(arg1, shape=shape)) File "/usr/local/lib/python2.7/dist-packages/scipy/sparse/coo.py", line 150, in init self._shape = check_shape((M, N)) File "/usr/local/lib/python2.7/dist-packages/scipy/sparse/sputils.py", line 281, in check_shape new_shape = tuple(operator.index(arg) for arg in args) File "/usr/local/lib/python2.7/dist-packages/scipy/sparse/sputils.py", line 281, in new_shape = tuple(operator.index(arg) for arg in args) TypeError: 'numpy.float64' object cannot be interpreted as an index ub16hp@UB16HP:~/ub16_prj/FMG$

[cGy147]

ub16hp@UB16HP:/ub16_prj/FMG$ gcc -fPIC --shared setVal.c -o setVal.so ub16hp@UB16HP:/ub16_prj/FMG$ gcc -fPIC --shared partXY.c -o partXY.so ub16hp@UB16HP:/ub16_prj/FMG$ python mf_features_generator.py yelp-50k all 1 data: data/yelp-50k/exp_split/1/, path_str: all finish load data from data/yelp-50k/exp_split/1/sim_res/path_count/UPBCatB_top500.res, cost 32.46 seconds, users: 13663, items=8133 start generate mf features, (K, eps, reg, iters) = (10, 10, 10, 500) Traceback (most recent call last): File "mf_features_generator.py", line 165, in run_all_yelp() File "mf_features_generator.py", line 91, in run_all_yelp run(path_str) File "mf_features_generator.py", line 49, in run U,V = mf.run() File "/home/ub16hp/ub16_prj/FMG/mf.py", line 122, in run X = cm((self.data[:,2], (self.data[:,0], self.data[:,1]))) #index starting from 0 File "/usr/local/lib/python2.7/dist-packages/scipy/sparse/compressed.py", line 51, in init other = self.class(coo_matrix(arg1, shape=shape)) File "/usr/local/lib/python2.7/dist-packages/scipy/sparse/coo.py", line 150, in init self._shape = check_shape((M, N)) File "/usr/local/lib/python2.7/dist-packages/scipy/sparse/sputils.py", line 281, in check_shape new_shape = tuple(operator.index(arg) for arg in args) File "/usr/local/lib/python2.7/dist-packages/scipy/sparse/sputils.py", line 281, in new_shape = tuple(operator.index(arg) for arg in args) TypeError: 'numpy.float64' object cannot be interpreted as an index ub16hp@UB16HP:/ub16_prj/FMG$

I have the same problem.Maybe we need all the dependent lib versions.

[hzhaoaf]

@newfish12314 can you tell me your environment, like OS, python version, g++ version, etc?

[cGy147]

@newfish12314 can you tell me your environment, like OS, python version, g++ version, etc?

My environment: Python 2.7.16 |Anaconda, Inc.| [GCC 7.3.0] on linux2,

I think the problem lies in the file"mf.py", line 122, X = cm((self.data[:,2], (self.data[:,0], self.data[:,1])))
In the function csr_matrix as cm ,the index must be integer . And in the file "mf_features_generator.py",line 36, data = np.loadtxt(sim_filename) In the function loadtxt ,the default dtype is np.float64.

If I change the dtype of data , when I run "python mf_features_generator.py yelp-50k URNSRUB 1",I encounter the bug :

data: data/yelp-50k/exp_split/1/, path_str: URNSRUB finish load data from data/yelp-50k/exp_split/1/sim_res/path_count/URNSRUB_top500.res, cost 0.78 seconds, users: 2283, items=3562 start generate mf features, (K, eps, reg, iters) = (10, 10, 10, 500) Traceback (most recent call last): File "mf_features_generator.py", line 183, in run(path_str) File "mf_features_generator.py", line 53, in run U,V = mf.run() File "/home/chy/FMG_master/object/FMG-master/mf.py", line 135, in run obs = omega.copy().data.astype(np.float64).reshape(self.train_num, 1, -1) ValueError: total size of new array must be unchanged

I tried to install numpy 1.11.0 ,but it didn't work. So I guess maybe I have a wrong scipy version or other dependent lib versions.

[hzhaoaf]

@newfish12314 I guess the problem may lie in the type of data[:,0] and data[:,1] in "X = cm((self.data[:,2], (self.data[:,0], self.data[:,1])))",

With np.loadtxt, the data type is float, while to construct the csr_matrix, the index should be integer, then you may try to change the type self.data[:,0].astype(np.int32) like this.

Besides, my numpy is 1.14.0. And for your convenience, I also updated the "so" files in my server to this project repository (checked the latest commit), which are "setVal.so" and "partXY.so", but I am not sure whether it can run in your server. You may try it.

[cGy147]

@newfish12314 I guess the problem may lie in the type of data[:,0] and data[:,1] in "X = cm((self.data[:,2], (self.data[:,0], self.data[:,1])))",

With np.loadtxt, the data type is float, while to construct the csr_matrix, the index should be integer, then you may try to change the type self.data[:,0].astype(np.int32) like this.

Besides, my numpy is 1.14.0. And for your convenience, I also updated the "so" files in my server to this project repository (checked the latest commit), which are "setVal.so" and "partXY.so", but I am not sure whether it can run in your server. You may try it.

Thank you very much for your reply. I change the code : X = cm((self.data[:,2], (self.data[:,0].astype(np.int32) , self.data[:,1].astype(np.int32) ))) And I updated my "so"file, When I run "python mf_features_generator.py yelp-50k UNBUB 1" ,the code works. So I think the "setVal.so" and "partXY.so"can run in my server.

But when I run "python mf_features_generator.py yelp-50k URNSRUB 1",I encounter the bug :

finish load data from data/yelp-50k/exp_split/1/sim_res/path_count/URNSRUB_top500.res, cost 1.27 seconds, users: 2283, items=3562 start generate mf features, (K, eps, reg, iters) = (10, 10, 10, 500) Traceback (most recent call last): File "mf_features_generator.py", line 183, in run(path_str) File "mf_features_generator.py", line 53, in run U,V = mf.run() File "/home/chy/FMG_master/object/FMG-master/mf.py", line 135, in run obs = omega.copy().data.astype(np.float64).reshape(self.train_num, 1) ValueError: cannot reshape array of size 27234 into shape (54468, 1)

I check the file"URNSRUB_top500.res", I just find the third column is float. Should I change the column to integer? I don't understand why the type of data could cause this bug.

[hzhaoaf]

@newfish12314 I think this error should be the number of data size, rather than the problems with data type. You may check in more detail.