search

pycaret / pycaret

An open-source, low-code machine learning library in Python
https://www.pycaret.org
MIT License
8.83k stars 1.76k forks source link

Error message installing to Jupyter Notebooks #118

Closed robert-hoyt closed 4 years ago

robert-hoyt commented 4 years ago

I was able to install Pycaret to CoLab but not Juptyer Notebooks via Anaconda. Pycaret will load and almost complete but each time regardless of how I install it (pip, conda) (terminal, Notebook) I get an error message
ERROR: Command errored out with exit status 1: command: /Applications/anaconda3/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/4c/zjb0c1753g70dgrrsb3xy6c80000gn/T/pip-install-qbhjy_h2/pycaret/setup.py'"'"'; file='"'"'/private/var/folders/4c/zjb0c1753g70dgrrsb3xy6c80000gn/T/pip-install-qbhjy_h2/pycaret/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info cwd: /private/var/folders/4c/zjb0c1753g70dgrrsb3xy6c80000gn/T/pip-install-qbhjy_h2/pycaret/ Complete output (8 lines): running egg_info creating pip-egg-info/pycaret.egg-info writing pip-egg-info/pycaret.egg-info/PKG-INFO writing dependency_links to pip-egg-info/pycaret.egg-info/dependency_links.txt writing requirements to pip-egg-info/pycaret.egg-info/requires.txt writing top-level names to pip-egg-info/pycaret.egg-info/top_level.txt writing manifest file 'pip-egg-info/pycaret.egg-info/SOURCES.txt' error: package directory 'pycaret' does not exist ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output

Please advice and thanks

pycaret commented 4 years ago

Can you give more details about your system, OS etc?

robert-hoyt commented 4 years ago

Sorry I didn't include that. I have a Mac Air 2019 laptop running 10.15.4 (Catalina)

DaGuT commented 4 years ago

Hi,

Yes, there was a problem with setup.py file (you can fix it yourself by deleting packages=['pycaret'] line from setup.py).

Feel free to install dev version that already installs well and have plenty of bugfixes: !pip install https://github.com/pycaret/pycaret/archive/dev-1.0.1.zip

Dont forget to close the issue after you try it and see that it works ;)

robert-hoyt commented 4 years ago

I did install the dev version (zip file) and the exact error once again appeared

robert-hoyt commented 4 years ago

I may be incorrect that it was the same error. Here is the most recent error message

ERROR: Command errored out with exit status 1: command: /Applications/anaconda3/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/4c/zjb0c1753g70dgrrsb3xy6c80000gn/T/pip-install-ghl0_roz/xgboost/setup.py'"'"'; file='"'"'/private/var/folders/4c/zjb0c1753g70dgrrsb3xy6c80000gn/T/pip-install-ghl0_roz/xgboost/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info cwd: /private/var/folders/4c/zjb0c1753g70dgrrsb3xy6c80000gn/T/pip-install-ghl0_roz/xgboost/ Complete output (27 lines): ++ pwd

DaGuT commented 4 years ago

Hm, that's trully weird, as latest version of xgboost comes precompiled for most of systems (and that error that you've described lastely was most common for 0.80, so I may assume that you did not use zip version of dev build that I have sent).

You have few options: Try to find version of xgboost that is precompiled for you pip install xgboost==VERSION where version can be: 1.1.0rc2 , 1.0.2 , 0.90 (try each) this will install that specific version of xgboost. After that install pycaret again with pip install https://github.com/pycaret/pycaret/archive/dev-1.0.1.zip (just copy-paste to terminal, no need to download it yourself)

or

you can install cmake to your system and install dev version of pycaret again.

pycaret commented 4 years ago

@DaGuT Hi.

Could it be because our requirements.txt still says xgboost>=0.9.

Do you think changing that will fix the problem? I can do commit now.

Thanks.

DaGuT commented 4 years ago

No, it was a problem when it was just ==. Those [old] versions are not compiled for some system and were causing errors installing xgboost due to missing compilers.

After we've changed it to xgboost to >= we load new stable version, the most recent version is now fully okay with everything.

Also if you had any version of xgboost yourself (like I was using the latest), it was forcefully downgraded to 0.80 (or 0.90 i dont remember). So it caused a trouble with my other package. So please dont force specific version :) Also, >= solved problem with recent macs (and there are a lot of them on MAC right now)

Anyway, we cant satisfy ALL systems with any specific version, as xgboost requires compilation from time to time. So let's just leave opportunity for them to install compiler and any version of xgboost they want.

robert-hoyt commented 4 years ago

When I listed the installed Python packages I am using with Anaconda, xgboost was not part of them. I did manually install xgboost version 0.9. When I then tried to install PyCaret using the zip dev file pip install https://github.com/pycaret/pycaret/archive/dev-1.0.1.zip

This is the error message

via Collecting xgboost>=0.90 (from pycaret==1.0.1) Using cached https://files.pythonhosted.org/packages/36/a5/703d93321f57048596217789be7c186304a33aff5b1c48c89597a546c65e/xgboost-1.0.2.tar.gz ERROR: Command errored out with exit status 1: command: /Applications/anaconda3/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/4c/zjb0c1753g70dgrrsb3xy6c80000gn/T/pip-install-e4g_lp09/xgboost/setup.py'"'"'; file='"'"'/private/var/folders/4c/zjb0c1753g70dgrrsb3xy6c80000gn/T/pip-install-e4g_lp09/xgboost/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info cwd: /private/var/folders/4c/zjb0c1753g70dgrrsb3xy6c80000gn/T/pip-install-e4g_lp09/xgboost/ Complete output (27 lines): ++ pwd

DaGuT commented 4 years ago

what the heck xD If you have installed version why does it even tries to install it again? Are you sure they are under the same enviroment? because >=0.90 surely satisfies your 0.90 version.

Nevertheless, install it with pip install xgboost==0.90 https://github.com/pycaret/pycaret/archive/dev-1.0.1.zip

This will surely keep your xgboost 0.90 version and force install pycaret with it

robert-hoyt commented 4 years ago

I did use the script above with the following error

Collecting https://github.com/pycaret/pycaret/archive/dev-1.0.1.zip Using cached https://github.com/pycaret/pycaret/archive/dev-1.0.1.zip Collecting xgboost==0.90 Using cached https://files.pythonhosted.org/packages/96/84/4e2cae6247f397f83d8adc5c2a2a0c5d7d790a14a4c7400ff6574586f589/xgboost-0.90.tar.gz ERROR: Command errored out with exit status 1: command: /Applications/anaconda3/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/4c/zjb0c1753g70dgrrsb3xy6c80000gn/T/pip-install-lqngtuug/xgboost/setup.py'"'"'; file='"'"'/private/var/folders/4c/zjb0c1753g70dgrrsb3xy6c80000gn/T/pip-install-lqngtuug/xgboost/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info cwd: /private/var/folders/4c/zjb0c1753g70dgrrsb3xy6c80000gn/T/pip-install-lqngtuug/xgboost/ Complete output (29 lines): ++ pwd

DaGuT commented 4 years ago

100% xgboost is installed under different enviroment

Install conda install -c conda-forge xgboost and then install pycaret again

DaGuT commented 4 years ago

or you can install xcode-select --install or sudo xcode-select --reset You have mac, dont you?

With this xgboost==0.90 will install properly (according to logs), so mentioned before pip install xgboost==0.90 https://github.com/pycaret/pycaret/archive/dev-1.0.1.zip should work

robert-hoyt commented 4 years ago

I had already tried the above scripts conda install -c forge xgboost and tried to install with both pip install pycaret and the dev package, both generating errors. I have a Mac

DaGuT commented 4 years ago

Oh, I thought "I did manually install xgboost version 0.9" meant that it worked :\

Also, if you installed it with conda, then it's surely installed under some enviroment. There pycaret should be installed properly 142%

Nevertheless, please try https://github.com/pycaret/pycaret/issues/118#issuecomment-626904522

DaGuT commented 4 years ago

If you have brew on you system you can install 'brew install cmake' and then install xgboost==1.0.2

robert-hoyt commented 4 years ago

The code you mentioned produces syntax errors

robert-hoyt commented 4 years ago

I installed xcode-select on my Mac and then a developer file. Using the terminal I installed PyCaret using the zip file, as before. It appeared to install ok, so I thought I would check to see by writing from pycaret.utils import version version()

This generated

ModuleNotFoundError Traceback (most recent call last)

in ----> 1 from pycaret.utils import version 2 version() ModuleNotFoundError: No module named 'pycaret'
DaGuT commented 4 years ago

wtf :D That's something new. I need to check newer commits that I might have missed...

Try install normal pycaret then (pip install pycaret), please. Anyway, xgboost was installed, which means it's already great.

If it complains about xgboost again, then pip install xgboost==VERSION pycaret, where version is your current version of xgboost

robert-hoyt commented 4 years ago

It downloaded multiple packages with only a few minor error messages. When I then asked for the from pycaret.utils import version version(). it said 1.0. I think it is installed. Is there any other script I can write to confirm it is installed ok? Many thanks

pycaret commented 4 years ago

@robert-hoyt

If you are using Jupyter notebook try running this:

from pycaret.datasets import get_data data = get_data('boston')

from pycaret.regression import * s = setup(data, target = 'medv', silent=True, session_id=123)

compare_models()

Let us know if it works?

robert-hoyt commented 4 years ago

That started PyCaret and the usual pipeline. It did stop and give the following message

OSError Traceback (most recent call last)

in 3 4 from pycaret.regression import * ----> 5 s = setup(data, target = 'medv', silent=True, session_id=123) 6 7 compare_models() /Applications/anaconda3/lib/python3.7/site-packages/pycaret/regression.py in setup(data, target, train_size, sampling, sample_estimator, categorical_features, categorical_imputation, ordinal_features, high_cardinality_features, high_cardinality_method, numeric_features, numeric_imputation, date_features, ignore_features, normalize, normalize_method, transformation, transformation_method, handle_unknown_categorical, unknown_categorical_method, pca, pca_method, pca_components, ignore_low_variance, combine_rare_levels, rare_level_threshold, bin_numeric_features, remove_outliers, outliers_threshold, remove_multicollinearity, multicollinearity_threshold, create_clusters, cluster_iter, polynomial_features, polynomial_degree, trigonometry_features, polynomial_threshold, group_features, group_names, feature_selection, feature_selection_threshold, feature_interaction, feature_ratio, interaction_threshold, transform_target, transform_target_method, session_id, silent, profile) 903 904 #import library --> 905 from pycaret import preprocess 906 907 data = preprocess.Preprocess_Path_One(train_data = data, /Applications/anaconda3/lib/python3.7/site-packages/pycaret/preprocess.py in 31 from sklearn.ensemble import RandomForestClassifier as rfc 32 from sklearn.ensemble import RandomForestRegressor as rfr ---> 33 from lightgbm import LGBMClassifier as lgbmc 34 from lightgbm import LGBMRegressor as lgbmr 35 import sys /Applications/anaconda3/lib/python3.7/site-packages/lightgbm/__init__.py in 6 from __future__ import absolute_import 7 ----> 8 from .basic import Booster, Dataset 9 from .callback import (early_stopping, print_evaluation, record_evaluation, 10 reset_parameter) /Applications/anaconda3/lib/python3.7/site-packages/lightgbm/basic.py in 31 32 ---> 33 _LIB = _load_lib() 34 35 /Applications/anaconda3/lib/python3.7/site-packages/lightgbm/basic.py in _load_lib() 26 if len(lib_path) == 0: 27 return None ---> 28 lib = ctypes.cdll.LoadLibrary(lib_path[0]) 29 lib.LGBM_GetLastError.restype = ctypes.c_char_p 30 return lib /Applications/anaconda3/lib/python3.7/ctypes/__init__.py in LoadLibrary(self, name) 440 441 def LoadLibrary(self, name): --> 442 return self._dlltype(name) 443 444 cdll = LibraryLoader(CDLL) /Applications/anaconda3/lib/python3.7/ctypes/__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error) 362 363 if handle is None: --> 364 self._handle = _dlopen(self._name, mode) 365 else: 366 self._handle = handle OSError: dlopen(/Applications/anaconda3/lib/python3.7/site-packages/lightgbm/lib_lightgbm.so, 6): Library not loaded: /usr/local/opt/libomp/lib/libomp.dylib Referenced from: /Applications/anaconda3/lib/python3.7/site-packages/lightgbm/lib_lightgbm.so Reason: image not found
robert-hoyt commented 4 years ago

The simple commands like head (), describe () and info () all work so I assume PyCaret is working properly in Jupyter Notebook

robert-hoyt commented 4 years ago

Many thanks. I'll close this string. Can't thank you enough

robert-hoyt commented 4 years ago

Many thanks to DaGuT for sticking by me until the issue was resolved

DaGuT commented 4 years ago

That started PyCaret and the usual pipeline. It did stop and give the following message

Ow, I also had that (or similar) problem.

@robert-hoyt As far as I remember I just installed pip install lightgbm, but then I was getting errors so do the following:

  1. install brew ( brew home page ). There is just 1 command, but I cant copy it here as my antivirus blocks the page the moment I paste it.
  2. install brew install libomp
  3. and then install pip install lightgbm again.

Sorry, I dont know any easier way to install libomp, so you need to install brew package manager :\

After that everything will work perfectly! (if you will have Out of memory error (watch your RAM)) then normalize your data (it's fixed in 1.0.1)

P.S. Ooof, good luck with your research! Let the further road have no rocks!