search

scikit-learn-contrib / DESlib

A Python library for dynamic classifier and ensemble selection
BSD 3-Clause "New" or "Revised" License
479 stars 106 forks source link

Problem with predicting and scoring when using Faiss KNN is being used for the knn calculation in the DS models #171

Closed sara-eb closed 4 years ago

sara-eb commented 4 years ago

Hi everyone,

I used Faiss method for knn calculation. Creating DS models and saving them was performed, and then the model was fitted into DSEL dataset. However, once I tried to do prediction and scoring for the test set, I am facing an error:

 score = knorae.score(X_test, y_test)
  File "/home/esara/deslib-env/lib/python3.6/site-packages/sklearn/base.py", line 357, in score
    return accuracy_score(y, self.predict(X), sample_weight=sample_weight)
  File "/home/esara/deslib-env/lib/python3.6/site-packages/deslib/base.py", line 440, in predict
    distances, neighbors = self._get_region_competence(X_DS)
  File "/home/esara/deslib-env/lib/python3.6/site-packages/deslib/base.py", line 381, in _get_region_competence
    return_distance=True)
  File "/home/esara/deslib-env/lib/python3.6/site-packages/deslib/util/faiss_knn_wrapper.py", line 112, in kneighbors
    dist, idx = self.index_.search(X, n_neighbors)
AttributeError: 'numpy.ndarray' object has no attribute 'search'

Do you have any idea about that? Your help and expert opinion is appreciated in advance.

Natlem commented 4 years ago

Hi,

Do you think we can have a look at your code ?

Thanks,

Menelau commented 4 years ago

@sara-eb

I just answered this question in the other thread: https://github.com/scikit-learn-contrib/DESlib/issues/167#issuecomment-553232299

sara-eb commented 4 years ago

@Menelau Thanks a lot

sara-eb commented 4 years ago

@Menelau Hi again, The models for KNORAU, KNORAE, and DESP could be saved and loaded successfully. However, the models for METADES and KNOP can not saved by the code (provided earlier) and it shows the following error message:

I followed the same procedure for saving and loading, what could be the reason?

Fitting METADES on X_DSEL dataset
[Parallel(n_jobs=6)]: Using backend ThreadingBackend with 6 concurrent workers.
[Parallel(n_jobs=6)]: Done  38 tasks      | elapsed:    3.1s
[Parallel(n_jobs=6)]: Done 188 tasks      | elapsed:   14.0s
[Parallel(n_jobs=6)]: Done 200 out of 200 | elapsed:   14.9s finished
[Parallel(n_jobs=6)]: Using backend ThreadingBackend with 6 concurrent workers.
[Parallel(n_jobs=6)]: Done  38 tasks      | elapsed:    2.5s
[Parallel(n_jobs=6)]: Done 188 tasks      | elapsed:   11.7s
[Parallel(n_jobs=6)]: Done 200 out of 200 | elapsed:   12.4s finished
Saving the dynamic selection model in  ./Dynamic_Selection_Models/
Traceback (most recent call last):
  File "/home/esara/PyCharm_myProjects/CreatePoolOfClassifiers/create_homogeneous_pool.py", line 396, in <module>
    main()
  File "/home/esara//PyCharm_myProjects/CreatePoolOfClassifiers/create_homogeneous_pool.py", line 366, in main
    knn_type='faiss')
  File "/home/esara/PyCharm_myProjects/CreatePoolOfClassifiers/create_homogeneous_pool.py", line 215, in run_metades
    save_ds(metades, outfile)
  File "/home/essara/PyCharm_myProjects/CreatePoolOfClassifiers/create_homogeneous_pool.py", line 68, in save_ds
    dill.dump(dsalgo, f)
  File "/home/esara/deslib-env/lib/python3.6/site-packages/dill/_dill.py", line 259, in dump
    Pickler(file, protocol, **_kwds).dump(obj)
  File "/home/esara/deslib-env/lib/python3.6/site-packages/dill/_dill.py", line 445, in dump
    StockPickler.dump(self, obj)
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 409, in dump
    self.save(obj)
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 521, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 634, in save_reduce
    save(state)
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/home/esara/deslib-env/lib/python3.6/site-packages/dill/_dill.py", line 912, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 847, in _batch_setitems
    save(v)
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 521, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 634, in save_reduce
    save(state)
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/home/esara/deslib-env/lib/python3.6/site-packages/dill/_dill.py", line 912, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 847, in _batch_setitems
    save(v)
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 521, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 634, in save_reduce
    save(state)
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/home/esara/deslib-env/lib/python3.6/site-packages/dill/_dill.py", line 912, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 852, in _batch_setitems
    save(v)
  File "/usr/local/python/3.6.2-static/lib/python3.6/pickle.py", line 496, in save
    rv = reduce(self.proto)
TypeError: can't pickle SwigPyObject objects

Thanks a lot

Menelau commented 4 years ago

@sara-eb Hello,

I'm sorry I forgot about these two cases (META-DES and KNOP) when I gave you the code for saving and loading DS methods. These methods uses the KNN method in a different way so they required an special handling.

I just updated the gist with the code for saving/loading DS methods to also handle this scenario: https://gist.github.com/Menelau/0cde51c3622be6313fd96b4dffb17996 only a minor change was required.

Thanks for reporting this error!

sara-eb commented 4 years ago

@Menelau Thanks a lot sir for this. Since I am working on your library by faiss kNN, I will inform you here if there is any.

Thanks once again for solving the issue