search

scikit-learn-contrib / lightning

Large-scale linear classification, regression and ranking in Python
https://contrib.scikit-learn.org/lightning/
1.73k stars 214 forks source link

Installation with python 3.7 #132

Closed zeeraktalat closed 4 years ago

zeeraktalat commented 5 years ago

Hey folks,

Installing this wit python 3.7 results in some errors which seem to be Cython related (see: https://github.com/pandas-dev/pandas/issues/21785, https://github.com/pandas-dev/pandas/issues/21785, https://github.com/scikit-learn-contrib/py-earth/issues/191).

Trying to install from source doesn't seem to work either for whatever reason.

The issue occurs on both OSX (10.14 - python 3.7.2) and Ubuntu (14.04 - python 3.7.1) 


    lightning/impl/randomkit/random_fast.cpp:5139:21: error: ‘PyThreadState’ has no member named ‘exc_type’
         *type = tstate->exc_type;
                         ^
    lightning/impl/randomkit/random_fast.cpp:5140:22: error: ‘PyThreadState’ has no member named ‘exc_value’
         *value = tstate->exc_value;
                          ^
    lightning/impl/randomkit/random_fast.cpp:5141:19: error: ‘PyThreadState’ has no member named ‘exc_traceback’
         *tb = tstate->exc_traceback;
                       ^
    lightning/impl/randomkit/random_fast.cpp: In function ‘void __Pyx_ExceptionReset(PyObject*, PyObject*, PyObject*)’:
    lightning/impl/randomkit/random_fast.cpp:5153:24: error: ‘PyThreadState’ has no member named ‘exc_type’
         tmp_type = tstate->exc_type;
                            ^
    lightning/impl/randomkit/random_fast.cpp:5154:25: error: ‘PyThreadState’ has no member named ‘exc_value’
         tmp_value = tstate->exc_value;
                             ^
    lightning/impl/randomkit/random_fast.cpp:5155:22: error: ‘PyThreadState’ has no member named ‘exc_traceback’
         tmp_tb = tstate->exc_traceback;
                          ^
    lightning/impl/randomkit/random_fast.cpp:5156:13: error: ‘PyThreadState’ has no member named ‘exc_type’
         tstate->exc_type = type;
                 ^
    lightning/impl/randomkit/random_fast.cpp:5157:13: error: ‘PyThreadState’ has no member named ‘exc_value’
         tstate->exc_value = value;
                 ^
    lightning/impl/randomkit/random_fast.cpp:5158:13: error: ‘PyThreadState’ has no member named ‘exc_traceback’
         tstate->exc_traceback = tb;
                 ^
    lightning/impl/randomkit/random_fast.cpp: In function ‘int __Pyx_GetException(PyObject**, PyObject**, PyObject**)’:
    lightning/impl/randomkit/random_fast.cpp:5201:24: error: ‘PyThreadState’ has no member named ‘exc_type’
         tmp_type = tstate->exc_type;
                            ^
    lightning/impl/randomkit/random_fast.cpp:5202:25: error: ‘PyThreadState’ has no member named ‘exc_value’
         tmp_value = tstate->exc_value;
                             ^
    lightning/impl/randomkit/random_fast.cpp:5203:22: error: ‘PyThreadState’ has no member named ‘exc_traceback’
         tmp_tb = tstate->exc_traceback;
                          ^
    lightning/impl/randomkit/random_fast.cpp:5204:13: error: ‘PyThreadState’ has no member named ‘exc_type’
         tstate->exc_type = local_type;
                 ^
    lightning/impl/randomkit/random_fast.cpp:5205:13: error: ‘PyThreadState’ has no member named ‘exc_value’
         tstate->exc_value = local_value;
                 ^
    lightning/impl/randomkit/random_fast.cpp:5206:13: error: ‘PyThreadState’ has no member named ‘exc_traceback’
         tstate->exc_traceback = local_tb;```
vene commented 5 years ago

Due to some CPython changes, the cppfiles in randomkit/ need to be regenerated with a recent cython. Would you mind doing so and submitting a PR? Thanks!

zeeraktalat commented 5 years ago

Done @vene. :)

selvamshan commented 5 years ago

Running setup.py install for sklearn-contrib-lightning ... error ERROR: Complete output from command /data/dsenv/bin/python3 -u -c 'import setuptools, tokenize;file='"'"'/tmp/pip-install-94tpbsr9/sklearn-contrib-lightning/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-lomn90hi/install-record.txt --single-version-externally-managed --compile --install-headers /data/dsenv/include/site/python3.7/sklearn-contrib-lightning: ERROR: Warning: Assuming default configuration (lightning/impl/datasets/{setup_datasets,setup}.py was not found)Appending lightning.impl.datasets configuration to lightning.impl Ignoring attempt to set 'name' (from 'lightning.impl' to 'lightning.impl.datasets') Warning: Assuming default configuration (lightning/impl/randomkit/tests/{setup_tests,setup}.py was not found)Appending lightning.impl.randomkit.tests configuration to lightning.impl.randomkit Ignoring attempt to set 'name' (from 'lightning.impl.randomkit' to 'lightning.impl.randomkit.tests') Appending lightning.impl.randomkit configuration to lightning.impl Ignoring attempt to set 'name' (from 'lightning.impl' to 'lightning.impl.randomkit') Warning: Assuming default configuration (lightning/impl/tests/{setup_tests,setup}.py was not found)Appending lightning.impl.tests configuration to lightning.impl Ignoring attempt to set 'name' (from 'lightning.impl' to 'lightning.impl.tests') Appending lightning.impl configuration to lightning Ignoring attempt to set 'name' (from 'lightning' to 'lightning.impl') C compiler: x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC

compile options: '-c'
extra options: '-fopenmp'
x86_64-linux-gnu-gcc: test_openmp.c
x86_64-linux-gnu-gcc -pthread objects/test_openmp.o -o test_openmp -fopenmp
Appending lightning configuration to
Ignoring attempt to set 'name' (from '' to 'lightning')
running install
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building extension "lightning.impl.adagrad_fast" sources
building extension "lightning.impl.dataset_fast" sources
building extension "lightning.impl.dual_cd_fast" sources
building extension "lightning.impl.loss_fast" sources
building extension "lightning.impl.prank_fast" sources
building extension "lightning.impl.primal_cd_fast" sources
building extension "lightning.impl.prox_fast" sources
building extension "lightning.impl.sag_fast" sources
building extension "lightning.impl.sdca_fast" sources
building extension "lightning.impl.sgd_fast" sources
building extension "lightning.impl.svrg_fast" sources
building extension "lightning.impl.randomkit.random_fast" sources
building data_files sources
build_src: building npy-pkg config files
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/lightning
copying lightning/regression.py -> build/lib.linux-x86_64-3.7/lightning
copying lightning/_build_utils.py -> build/lib.linux-x86_64-3.7/lightning
copying lightning/setup.py -> build/lib.linux-x86_64-3.7/lightning
copying lightning/ranking.py -> build/lib.linux-x86_64-3.7/lightning
copying lightning/datasets.py -> build/lib.linux-x86_64-3.7/lightning
copying lightning/classification.py -> build/lib.linux-x86_64-3.7/lightning
copying lightning/__init__.py -> build/lib.linux-x86_64-3.7/lightning
creating build/lib.linux-x86_64-3.7/lightning/impl
copying lightning/impl/primal_cd.py -> build/lib.linux-x86_64-3.7/lightning/impl
copying lightning/impl/fista.py -> build/lib.linux-x86_64-3.7/lightning/impl
copying lightning/impl/sgd.py -> build/lib.linux-x86_64-3.7/lightning/impl
copying lightning/impl/penalty.py -> build/lib.linux-x86_64-3.7/lightning/impl
copying lightning/impl/prank.py -> build/lib.linux-x86_64-3.7/lightning/impl
copying lightning/impl/dual_cd.py -> build/lib.linux-x86_64-3.7/lightning/impl
copying lightning/impl/setup.py -> build/lib.linux-x86_64-3.7/lightning/impl
copying lightning/impl/svrg.py -> build/lib.linux-x86_64-3.7/lightning/impl
copying lightning/impl/sag.py -> build/lib.linux-x86_64-3.7/lightning/impl
copying lightning/impl/base.py -> build/lib.linux-x86_64-3.7/lightning/impl
copying lightning/impl/adagrad.py -> build/lib.linux-x86_64-3.7/lightning/impl
copying lightning/impl/primal_newton.py -> build/lib.linux-x86_64-3.7/lightning/impl
copying lightning/impl/__init__.py -> build/lib.linux-x86_64-3.7/lightning/impl
copying lightning/impl/sdca.py -> build/lib.linux-x86_64-3.7/lightning/impl
creating build/lib.linux-x86_64-3.7/lightning/impl/datasets
copying lightning/impl/datasets/loaders.py -> build/lib.linux-x86_64-3.7/lightning/impl/datasets
copying lightning/impl/datasets/samples_generator.py -> build/lib.linux-x86_64-3.7/lightning/impl/datasets
copying lightning/impl/datasets/__init__.py -> build/lib.linux-x86_64-3.7/lightning/impl/datasets
copying lightning/impl/datasets/utils.py -> build/lib.linux-x86_64-3.7/lightning/impl/datasets
creating build/lib.linux-x86_64-3.7/lightning/impl/randomkit
copying lightning/impl/randomkit/setup.py -> build/lib.linux-x86_64-3.7/lightning/impl/randomkit
copying lightning/impl/randomkit/__init__.py -> build/lib.linux-x86_64-3.7/lightning/impl/randomkit
creating build/lib.linux-x86_64-3.7/lightning/impl/randomkit/tests
copying lightning/impl/randomkit/tests/test_random.py -> build/lib.linux-x86_64-3.7/lightning/impl/randomkit/tests
copying lightning/impl/randomkit/tests/__init__.py -> build/lib.linux-x86_64-3.7/lightning/impl/randomkit/tests
creating build/lib.linux-x86_64-3.7/lightning/impl/tests
copying lightning/impl/tests/test_sgd.py -> build/lib.linux-x86_64-3.7/lightning/impl/tests
copying lightning/impl/tests/test_svrg.py -> build/lib.linux-x86_64-3.7/lightning/impl/tests
copying lightning/impl/tests/test_sdca.py -> build/lib.linux-x86_64-3.7/lightning/impl/tests
copying lightning/impl/tests/test_sag.py -> build/lib.linux-x86_64-3.7/lightning/impl/tests
copying lightning/impl/tests/test_prank.py -> build/lib.linux-x86_64-3.7/lightning/impl/tests
copying lightning/impl/tests/test_adagrad.py -> build/lib.linux-x86_64-3.7/lightning/impl/tests
copying lightning/impl/tests/test_penalty.py -> build/lib.linux-x86_64-3.7/lightning/impl/tests
copying lightning/impl/tests/test_dataset.py -> build/lib.linux-x86_64-3.7/lightning/impl/tests
copying lightning/impl/tests/test_primal_newton.py -> build/lib.linux-x86_64-3.7/lightning/impl/tests
copying lightning/impl/tests/test_dual_cd.py -> build/lib.linux-x86_64-3.7/lightning/impl/tests
copying lightning/impl/tests/test_fista.py -> build/lib.linux-x86_64-3.7/lightning/impl/tests
copying lightning/impl/tests/test_primal_cd.py -> build/lib.linux-x86_64-3.7/lightning/impl/tests
copying lightning/impl/tests/__init__.py -> build/lib.linux-x86_64-3.7/lightning/impl/tests
copying lightning/impl/tests/test_prox.py -> build/lib.linux-x86_64-3.7/lightning/impl/tests
copying lightning/impl/tests/utils.py -> build/lib.linux-x86_64-3.7/lightning/impl/tests
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
building 'lightning.impl.adagrad_fast' extension
compiling C++ sources
C compiler: x86_64-linux-gnu-g++ -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC

creating build/temp.linux-x86_64-3.7/lightning
creating build/temp.linux-x86_64-3.7/lightning/impl
compile options: '-I/data/dsenv/lib/python3.7/site-packages/numpy/core/include -I/tmp/pip-install-94tpbsr9/sklearn-contrib-lightning/lightning/impl/randomkit -I/data/dsenv/lib/python3.7/site-packages/numpy/core/include -I/data/dsenv/include -I/usr/include/python3.7m -c'
x86_64-linux-gnu-g++: lightning/impl/adagrad_fast.cpp
In file included from /data/dsenv/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1822,
                 from /data/dsenv/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from /data/dsenv/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                 from lightning/impl/adagrad_fast.cpp:466:
/data/dsenv/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
 #warning "Using deprecated NumPy API, disable it with " \
  ^~~~~~~
lightning/impl/adagrad_fast.cpp: In function ‘PyObject* __Pyx_PyCFunction_FastCall(PyObject*, PyObject**, Py_ssize_t)’:
lightning/impl/adagrad_fast.cpp:7284:69: error: too many arguments to function
     return (*((__Pyx_PyCFunctionFast)meth)) (self, args, nargs, NULL);
                                                                     ^
lightning/impl/adagrad_fast.cpp: In function ‘void __Pyx__ExceptionSave(PyThreadState*, PyObject**, PyObject**, PyObject**)’:
lightning/impl/adagrad_fast.cpp:7656:21: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_type’; did you mean ‘curexc_type’?
     *type = tstate->exc_type;
                     ^~~~~~~~
                     curexc_type
lightning/impl/adagrad_fast.cpp:7657:22: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_value’; did you mean ‘curexc_value’?
     *value = tstate->exc_value;
                      ^~~~~~~~~
                      curexc_value
lightning/impl/adagrad_fast.cpp:7658:19: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
     *tb = tstate->exc_traceback;
                   ^~~~~~~~~~~~~
                   curexc_traceback
lightning/impl/adagrad_fast.cpp: In function ‘void __Pyx__ExceptionReset(PyThreadState*, PyObject*, PyObject*, PyObject*)’:
lightning/impl/adagrad_fast.cpp:7665:24: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_type’; did you mean ‘curexc_type’?
     tmp_type = tstate->exc_type;
                        ^~~~~~~~
                        curexc_type
lightning/impl/adagrad_fast.cpp:7666:25: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_value’; did you mean ‘curexc_value’?
     tmp_value = tstate->exc_value;
                         ^~~~~~~~~
                         curexc_value
lightning/impl/adagrad_fast.cpp:7667:22: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
     tmp_tb = tstate->exc_traceback;
                      ^~~~~~~~~~~~~
                      curexc_traceback
lightning/impl/adagrad_fast.cpp:7668:13: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_type’; did you mean ‘curexc_type’?
     tstate->exc_type = type;
             ^~~~~~~~
             curexc_type
lightning/impl/adagrad_fast.cpp:7669:13: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_value’; did you mean ‘curexc_value’?
     tstate->exc_value = value;
             ^~~~~~~~~
             curexc_value
lightning/impl/adagrad_fast.cpp:7670:13: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
     tstate->exc_traceback = tb;
             ^~~~~~~~~~~~~
             curexc_traceback
lightning/impl/adagrad_fast.cpp: In function ‘int __Pyx__GetException(PyThreadState*, PyObject**, PyObject**, PyObject**)’:
lightning/impl/adagrad_fast.cpp:7725:24: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_type’; did you mean ‘curexc_type’?
     tmp_type = tstate->exc_type;
                        ^~~~~~~~
                        curexc_type
lightning/impl/adagrad_fast.cpp:7726:25: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_value’; did you mean ‘curexc_value’?
     tmp_value = tstate->exc_value;
                         ^~~~~~~~~
                         curexc_value
lightning/impl/adagrad_fast.cpp:7727:22: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
     tmp_tb = tstate->exc_traceback;
                      ^~~~~~~~~~~~~
                      curexc_traceback
lightning/impl/adagrad_fast.cpp:7728:13: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_type’; did you mean ‘curexc_type’?
     tstate->exc_type = local_type;
             ^~~~~~~~
             curexc_type
lightning/impl/adagrad_fast.cpp:7729:13: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_value’; did you mean ‘curexc_value’?
     tstate->exc_value = local_value;
             ^~~~~~~~~
             curexc_value
lightning/impl/adagrad_fast.cpp:7730:13: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
     tstate->exc_traceback = local_tb;
             ^~~~~~~~~~~~~
             curexc_traceback
error: Command "x86_64-linux-gnu-g++ -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/data/dsenv/lib/python3.7/site-packages/numpy/core/include -I/tmp/pip-install-94tpbsr9/sklearn-contrib-lightning/lightning/impl/randomkit -I/data/dsenv/lib/python3.7/site-packages/numpy/core/include -I/data/dsenv/include -I/usr/include/python3.7m -c lightning/impl/adagrad_fast.cpp -o build/temp.linux-x86_64-3.7/lightning/impl/adagrad_fast.o -MMD -MF build/temp.linux-x86_64-3.7/lightning/impl/adagrad_fast.o.d" failed with exit status 1
----------------------------------------

ERROR: Command "/data/dsenv/bin/python3 -u -c 'import setuptools, tokenize;file='"'"'/tmp/pip-install-94tpbsr9/sklearn-contrib-lightning/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-lomn90hi/install-record.txt --single-version-externally-managed --compile --install-headers /data/dsenv/include/site/python3.7/sklearn-contrib-lightning" failed with error code 1 in /tmp/pip-install-94tpbsr9/sklearn-contrib-lightning/

barocsi commented 4 years ago

Same here, did you manage to resolve this?

zeeraktalat commented 4 years ago

@barocsi My issue should have been fixed with #133. I'm not sure about @selvamshan's issue which looks to be different.

barocsi commented 4 years ago

Thanks! @ZeerakW

vene commented 4 years ago

this reminds me now it's time to test & build wheels for python 3.8 too :)

maybe in a couple of days..

mohataher commented 4 years ago

Has this been fixed yet?

zeeraktalat commented 4 years ago

@mohataher This should be fixed in #133