hyperopt / hyperopt-sklearn

Hyper-parameter optimization for sklearn
hyperopt.github.io/hyperopt-sklearn
Other
1.58k stars 272 forks source link

No module named "hpsklearn.estimator" #185

Closed LarissaHolm closed 2 years ago

LarissaHolm commented 2 years ago

Hello everyone, I'm trying to use Hyperopt-Sklearn in Google Colab. However, after using "!pip install hpsklearn", I get the following error message when trying to import the estimator through "from hpsklearn import HyperoptEstimator": ModuleNotFoundError Traceback (most recent call last) in () 1 get_ipython().system('pip install hpsklearn') ----> 2 from hpsklearn import HyperoptEstimator 3 from hpsklearn import any_classifier 4 from hpsklearn import any_preprocessing 5 from hyperopt import tpe

/usr/local/lib/python3.7/dist-packages/hpsklearn/init.py in () ----> 1 from .estimator import hyperopt_estimator as HyperoptEstimator 2 from .components import * # noqa 3 from .components.multiclass import \ 4 one_vs_rest_classifier, \ 5 one_vs_one_classifier, \

ModuleNotFoundError: No module named 'hpsklearn.estimator'

I have also tried loading the Estimator by accessing the github through "!pip install git+https://github.com/hyperopt/hyperopt-sklearn", but get the same error message.

I have the following packages/libraries installed before trying to work with Hyperopt: !pip install imgaug==0.2.5 !pip install --upgrade scipy !pip install --upgrade sklearn !sudo apt-get install build-essential swig !pip install auto-sklearn import autosklearn.classification import autosklearn.regression import numpy as np import random import pandas as pd import sklearn.datasets import sklearn.metrics import scipy import matplotlib.pyplot as plt import csv import pprint as pprint from sklearn.model_selection import train_test_split from sklearn.model_selection import RepeatedKFold from sklearn.model_selection import RepeatedStratifiedKFold

Is anybody else having this issue and if so, is there a solution or workaround? I'm fairly new to AutoML and its programming.

mandjevant commented 2 years ago

Hey there!

The pipy package might not be up to date. Could you try installing it like it's mentioned in the Readme? pip install git+https://github.com/hyperopt/hyperopt-sklearn

Let me know if you run into any issues!

LarissaHolm commented 2 years ago

Thank you for the fast response! I already tried that and the result remains the same even after restarting the runtime.

Collecting git+https://github.com/hyperopt/hyperopt-sklearn Cloning https://github.com/hyperopt/hyperopt-sklearn to /tmp/pip-req-build-l8id_vgr Running command git clone -q https://github.com/hyperopt/hyperopt-sklearn /tmp/pip-req-build-l8id_vgr Installing build dependencies ... done Getting requirements to build wheel ... done Preparing wheel metadata ... done Requirement already satisfied: scipy>=1.7.1 in /usr/local/lib/python3.7/dist-packages (from hpsklearn==1.0.3) (1.7.3) Requirement already satisfied: numpy>=1.21.2 in /usr/local/lib/python3.7/dist-packages (from hpsklearn==1.0.3) (1.21.6) Requirement already satisfied: hyperopt>=0.2.5 in /usr/local/lib/python3.7/dist-packages (from hpsklearn==1.0.3) (0.2.7) Requirement already satisfied: scikit-learn>=1.0 in /usr/local/lib/python3.7/dist-packages (from hpsklearn==1.0.3) (1.0.2) Requirement already satisfied: py4j in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->hpsklearn==1.0.3) (0.10.9.5) Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->hpsklearn==1.0.3) (4.64.0) Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->hpsklearn==1.0.3) (1.15.0) Requirement already satisfied: networkx>=2.2 in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->hpsklearn==1.0.3) (2.6.3) Requirement already satisfied: cloudpickle in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->hpsklearn==1.0.3) (2.0.0) Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->hpsklearn==1.0.3) (0.16.0) Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=1.0->hpsklearn==1.0.3) (3.1.0) Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=1.0->hpsklearn==1.0.3) (1.1.0)

ModuleNotFoundError Traceback (most recent call last) in () 2 #import hpsklearn 3 get_ipython().system('pip install git+https://github.com/hyperopt/hyperopt-sklearn') ----> 4 from hpsklearn import HyperoptEstimator 5 from hpsklearn import any_classifier 6 from hpsklearn import any_preprocessing

/usr/local/lib/python3.7/dist-packages/hpsklearn/init.py in () ----> 1 from .estimator import hyperopt_estimator as HyperoptEstimator 2 from .components import * # noqa 3 from .components.multiclass import \ 4 one_vs_rest_classifier, \ 5 one_vs_one_classifier, \

ModuleNotFoundError: No module named 'hpsklearn.estimator'

PhilippEberl commented 2 years ago

I have the exact same error, nothing seems to fix it.

mandjevant commented 2 years ago

I see, I get this error too when I'm installing in Jupyter. Looking at the package it installed, it only downloads the __init__ file.

I apologize for this oversight and aim to fix the issue and come with a solution within 48 hours.

Bouchenemehdi24 commented 2 years ago

I have the same issue. Iam working with Google colab and after the end of the installation i have the following message: ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.

After from hpsklearn import HyperoptEstimator

ModuleNotFoundError: No module named 'hpsklearn.estimator'

Thanks in advance.

mandjevant commented 2 years ago

@LarissaHolm @PhilippEberl @Bouchenemehdi24 Thank you for raising this issue. I determined what the mistake was and I suggested a fix in #186

When it gets merged (maximum 24 hours) I will close this issue.

If you require to use hpsklearn right now, please use pip install git+https://github.com/mandjevant/hyperopt-sklearn.

This has the fix and should work now.

Once again my apologies.

LarissaHolm commented 2 years ago

Hello @mandjevant Thank you very much! I just tried the way you suggested, but am still getting the same error message in Colab:

!pip install git+https://github.com/mandjevant/hyperopt-sklearn from hpsklearn import HyperoptEstimator from hpsklearn import any_classifier from hpsklearn import any_preprocessing from hyperopt import tpe

Collecting git+https://github.com/mandjevant/hyperopt-sklearn Cloning https://github.com/mandjevant/hyperopt-sklearn to /tmp/pip-req-build-csk9l733 Running command git clone -q https://github.com/mandjevant/hyperopt-sklearn /tmp/pip-req-build-csk9l733 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing wheel metadata ... done Requirement already satisfied: scipy>=1.7.1 in /usr/local/lib/python3.7/dist-packages (from hpsklearn==1.0.3) (1.7.3) Requirement already satisfied: numpy>=1.21.2 in /usr/local/lib/python3.7/dist-packages (from hpsklearn==1.0.3) (1.21.6) Requirement already satisfied: hyperopt>=0.2.5 in /usr/local/lib/python3.7/dist-packages (from hpsklearn==1.0.3) (0.2.7) Requirement already satisfied: scikit-learn>=1.0 in /usr/local/lib/python3.7/dist-packages (from hpsklearn==1.0.3) (1.0.2) Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->hpsklearn==1.0.3) (1.15.0) Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->hpsklearn==1.0.3) (0.16.0) Requirement already satisfied: py4j in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->hpsklearn==1.0.3) (0.10.9.5) Requirement already satisfied: networkx>=2.2 in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->hpsklearn==1.0.3) (2.6.3) Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->hpsklearn==1.0.3) (4.64.0) Requirement already satisfied: cloudpickle in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->hpsklearn==1.0.3) (1.3.0) Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=1.0->hpsklearn==1.0.3) (1.1.0) Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=1.0->hpsklearn==1.0.3) (3.1.0)

ModuleNotFoundError Traceback (most recent call last) in () 1 get_ipython().system('pip install git+https://github.com/mandjevant/hyperopt-sklearn') ----> 2 from hpsklearn import HyperoptEstimator 3 from hpsklearn import any_classifier 4 from hpsklearn import any_preprocessing 5 from hyperopt import tpe

/usr/local/lib/python3.7/dist-packages/hpsklearn/init.py in () ----> 1 from .estimator import hyperopt_estimator as HyperoptEstimator 2 from .components import * # noqa 3 from .components.multiclass import \ 4 one_vs_rest_classifier, \ 5 one_vs_one_classifier, \

ModuleNotFoundError: No module named 'hpsklearn.estimator'

Is it a mistake I'm making or should I wait until the merge is complete?

mandjevant commented 2 years ago

You should probably manually remove the current hpsklearn directory and it's dist- directory. @LarissaHolm On windows it's in C:/Users/<Username>/Anaconda3/Envs/<EnvName>/Lib/site-packages/hpsklearn

mandjevant commented 2 years ago

Hey @LarissaHolm I just noticed I misread Colab as Conda.

I do think you must uninstall the previously installed hpsklearn module. I have just tested running !pip install git+https://github.com/mandjevant/hyperopt-sklearn on a clean google colab notebook and it runs without problems.

PhilippEberl commented 2 years ago

@mandjevant Thanks for the quick fix, though now I get a new error

      3 import matplotlib.pyplot as plt
      5 from hyperopt import tpe
----> 6 from hpsklearn import HyperoptEstimator, k_neighbors_regressor, linear_svr
      7 from sklearn.model_selection import cross_val_score, train_test_split

File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/hpsklearn/__init__.py:1, in <module>
----> 1 from .estimator import hyperopt_estimator as HyperoptEstimator
      2 from .components import *  # noqa
      3 from .components.multiclass import \
      4     one_vs_rest_classifier, \
      5     one_vs_one_classifier, \
      6     output_code_classifier

File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/hpsklearn/estimator/__init__.py:1, in <module>
----> 1 from .estimator import hyperopt_estimator

File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/hpsklearn/estimator/estimator.py:14, in <module>
     12 import scipy.sparse
     13 import numpy as np
---> 14 import numpy.typing as npt
     15 import pathlib
     16 import inspect

File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/numpy/typing/__init__.py:324, in <module>
    311 from ._scalars import (
    312     _CharLike_co,
    313     _BoolLike_co,
   (...)
    321     _VoidLike_co,
    322 )
    323 from ._shape import _Shape, _ShapeLike
--> 324 from ._dtype_like import (
    325     DTypeLike as DTypeLike,
    326     _SupportsDType,
    327     _VoidDTypeLike,
    328     _DTypeLikeBool,
    329     _DTypeLikeUInt,
    330     _DTypeLikeInt,
    331     _DTypeLikeFloat,
    332     _DTypeLikeComplex,
    333     _DTypeLikeTD64,
    334     _DTypeLikeDT64,
    335     _DTypeLikeObject,
    336     _DTypeLikeVoid,
    337     _DTypeLikeStr,
    338     _DTypeLikeBytes,
    339     _DTypeLikeComplex_co,
    340 )
    341 from ._array_like import (
    342     ArrayLike as ArrayLike,
    343     _ArrayLike,
   (...)
    358     _ArrayLikeBytes_co,
    359 )
    360 from ._generic_alias import (
    361     NDArray as NDArray,
    362     _DType,
    363     _GenericAlias,
    364 )

File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/numpy/typing/_dtype_like.py:16, in <module>
     13 import numpy as np
     15 from ._shape import _ShapeLike
---> 16 from ._generic_alias import _DType as DType
     18 from ._char_codes import (
     19     _BoolCodes,
     20     _UInt8Codes,
   (...)
     57     _ObjectCodes,
     58 )
     60 _DTypeLikeNested = Any  # TODO: wait for support for recursive types

File /opt/homebrew/Caskroom/miniforge/base/envs/tensorflow/lib/python3.9/site-packages/numpy/typing/_generic_alias.py:211, in <module>
    208 ScalarType = TypeVar("ScalarType", bound=np.generic, covariant=True)
    210 if TYPE_CHECKING or sys.version_info >= (3, 9):
--> 211     _DType = np.dtype[ScalarType]
    212     NDArray = np.ndarray[Any, np.dtype[ScalarType]]
    213 else:

TypeError: 'type' object is not subscriptable

I uninstalled the old version a did a reinstall with the new one, but maybe I am doing something wrong.

mandjevant commented 2 years ago

Hey @PhilippEberl I'm quite certain you should update numpy to fix this issue. Could you let me know what your current numpy version is?

PhilippEberl commented 2 years ago

@mandjevant I run on numpy v1.22.1

mandjevant commented 2 years ago

@mandjevant I run on numpy v1.22.1

I personally do not experience any problems with hpsklearn or numpy in v1.22.1 or other versions. I am confused as to why you get this error. I'd like to help you fix it.

Could you try removing your entire environment, then starting with a clean environment and see if you still get any issues?

If you do have the same issue, could you give me

With this information I can try and replicate your computer 1:1 to see if the issue is perhaps platform-specific.

Thank you and good luck!

PhilippEberl commented 2 years ago

@mandjevant I run on numpy v1.22.1

I personally do not experience any problems with hpsklearn or numpy in v1.22.1 or other versions. I am confused as to why you get this error. I'd like to help you fix it.

Could you try removing your entire environment, then starting with a clean environment and see if you still get any issues?

If you do have the same issue, could you give me

  • The pip list
  • And your OS and current version

With this information I can try and replicate your computer 1:1 to see if the issue is perhaps platform-specific.

Thank you and good luck!

I reinstalled all required packages, including hpsklearn and now the issue is resovled. All notebooks run normally.

Thanks!

mandjevant commented 2 years ago

Glad to hear this, thanks for letting me know @PhilippEberl