search

automl / fanova

Functional ANOVA
122 stars 45 forks source link

Support for Mac OSX? #74

Open janvanrijn opened 5 years ago

janvanrijn commented 5 years ago

Is the library supported on Mac systems? A student of mine, @abhinavs95, is having problems running fanova on a MAC (installing is no problem).

@abhinavs95, can you please share the stacktrace?

(If not, that would be good to know, as then we can arrange a linux machine for Abhinav)

AndreBiedenkapp commented 5 years ago

We've never tested it on Mac OSX, only Ubuntu 16.04, 18.04 or ArchLinux. Something I would recommend is to use at least Python3.4. Also, the pyrfr needs swig3.0. So the best thing would be to use conda to easily setup the environment.

Let me know if that solved the issue.

abhinavs95 commented 5 years ago

Hi @AndreBiedenkapp I'm pasting the stacktrace below for your reference. I face this error when I try to create plots using fanova visualizer.

I am already using python 3.6 and swig 3.0.12 in a conda environment. I tried both installing from source and using pip.

This is the initial error I faced:

Traceback (most recent call last):
  File "/anaconda3/envs/fanova/lib/python3.6/site-packages/pyrfr/regression.py", line 14, in swig_import_helper
    return importlib.import_module(mname)
  File "/anaconda3/envs/fanova/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 658, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 571, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 922, in create_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: dlopen(/anaconda3/envs/fanova/lib/python3.6/site-packages/pyrfr/_regression.cpython-36m-darwin.so, 2): Symbol not found: __ZNKSt5ctypeIcE13_M_widen_initEv
  Referenced from: /anaconda3/envs/fanova/lib/python3.6/site-packages/pyrfr/_regression.cpython-36m-darwin.so
  Expected in: /usr/lib/libstdc++.6.dylib
 in /anaconda3/envs/fanova/lib/python3.6/site-packages/pyrfr/_regression.cpython-36m-darwin.so

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "fanova-run-mnist.py", line 2, in <module>
    from fanova import fANOVA
  File "/anaconda3/envs/fanova/lib/python3.6/site-packages/fanova/__init__.py", line 1, in <module>
    from fanova.fanova import *
  File "/anaconda3/envs/fanova/lib/python3.6/site-packages/fanova/fanova.py", line 5, in <module>
    import pyrfr.regression as reg
  File "/anaconda3/envs/fanova/lib/python3.6/site-packages/pyrfr/regression.py", line 17, in <module>
    _regression = swig_import_helper()
  File "/anaconda3/envs/fanova/lib/python3.6/site-packages/pyrfr/regression.py", line 16, in swig_import_helper
    return importlib.import_module('_regression')
  File "/anaconda3/envs/fanova/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_regression'

I tried to fix it by installing gcc after which I get the following:

2018-11-14 13:26:01.168 python[895:20002] -[NSApplication _setup:]: unrecognized selector sent to instance 0x7ff168de3a30
2018-11-14 13:26:01.170 python[895:20002] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[NSApplication _setup:]: unrecognized selector sent to instance 0x7ff168de3a30'
*** First throw call stack:
(
    0   CoreFoundation                      0x00007fff328dd43d __exceptionPreprocess + 256
    1   libobjc.A.dylib                     0x00007fff5e7ea720 objc_exception_throw + 48
    2   CoreFoundation                      0x00007fff3295a255 -[NSObject(NSObject) __retain_OA] + 0
    3   CoreFoundation                      0x00007fff3287cad0 ___forwarding___ + 1486
    4   CoreFoundation                      0x00007fff3287c478 _CF_forwarding_prep_0 + 120
    5   libtk8.6.dylib                      0x000000011eba331d TkpInit + 413
    6   libtk8.6.dylib                      0x000000011eafb17e Initialize + 2622
    7   _tkinter.cpython-36m-darwin.so      0x000000011e923a16 _tkinter_create + 1174
    8   python                              0x000000010b5c2088 _PyCFunction_FastCallDict + 200
    9   python                              0x000000010b698f4f call_function + 143
    10  python                              0x000000010b696abf _PyEval_EvalFrameDefault + 46847
    11  python                              0x000000010b68a209 _PyEval_EvalCodeWithName + 425
    12  python                              0x000000010b699b1c _PyFunction_FastCallDict + 364
    13  python                              0x000000010b5408b0 _PyObject_FastCallDict + 320
    14  python                              0x000000010b567fe8 method_call + 136
    15  python                              0x000000010b547efe PyObject_Call + 62
    16  python                              0x000000010b5e9385 slot_tp_init + 117
    17  python                              0x000000010b5ed8c1 type_call + 241
    18  python                              0x000000010b540821 _PyObject_FastCallDict + 177
    19  python                              0x000000010b548a67 _PyObject_FastCallKeywords + 327
    20  python                              0x000000010b699048 call_function + 392
    21  python                              0x000000010b696b6f _PyEval_EvalFrameDefault + 47023
    22  python                              0x000000010b69930c fast_function + 188
    23  python                              0x000000010b698fac call_function + 236
    24  python                              0x000000010b696abf _PyEval_EvalFrameDefault + 46847
    25  python                              0x000000010b68a209 _PyEval_EvalCodeWithName + 425
    26  python                              0x000000010b699b1c _PyFunction_FastCallDict + 364
    27  python                              0x000000010b5408b0 _PyObject_FastCallDict + 320
    28  python                              0x000000010b567fe8 method_call + 136
    29  python                              0x000000010b547efe PyObject_Call + 62
    30  python                              0x000000010b696cc0 _PyEval_EvalFrameDefault + 47360
    31  python                              0x000000010b68a209 _PyEval_EvalCodeWithName + 425
    32  python                              0x000000010b6993ba fast_function + 362
    33  python                              0x000000010b698fac call_function + 236
    34  python                              0x000000010b696abf _PyEval_EvalFrameDefault + 46847
    35  python                              0x000000010b69930c fast_function + 188
    36  python                              0x000000010b698fac call_function + 236
    37  python                              0x000000010b696abf _PyEval_EvalFrameDefault + 46847
    38  python                              0x000000010b68a209 _PyEval_EvalCodeWithName + 425
    39  python                              0x000000010b6993ba fast_function + 362
    40  python                              0x000000010b698fac call_function + 236
    41  python                              0x000000010b696abf _PyEval_EvalFrameDefault + 46847
    42  python                              0x000000010b68a209 _PyEval_EvalCodeWithName + 425
    43  python                              0x000000010b6993ba fast_function + 362
    44  python                              0x000000010b698fac call_function + 236
    45  python                              0x000000010b696abf _PyEval_EvalFrameDefault + 46847
    46  python                              0x000000010b68a209 _PyEval_EvalCodeWithName + 425
    47  python                              0x000000010b6993ba fast_function + 362
    48  python                              0x000000010b698fac call_function + 236
    49  python                              0x000000010b696b6f _PyEval_EvalFrameDefault + 47023
    50  python                              0x000000010b68a209 _PyEval_EvalCodeWithName + 425
    51  python                              0x000000010b6e2d4c PyRun_FileExFlags + 252
    52  python                              0x000000010b6e2224 PyRun_SimpleFileExFlags + 372
    53  python                              0x000000010b708d66 Py_Main + 3734
    54  python                              0x000000010b538929 main + 313
    55  libdyld.dylib                       0x00007fff5f8b8085 start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException
Abort trap: 6
AndreBiedenkapp commented 5 years ago

Thank you for posting the trace.

You said that

[you] face this error when [you] try to create plots using fanova visualizer.

Does that mean that you can run the following minimal example?

from fanova import fANOVA
import numpy as np

X = np.random.randn(10, 2)
y = np.random.rand(10, 1)
f = fANOVA(X, y)
importances = f.quantify_importance([0, 1])
for k in importances:
    print(k, importances[k])

I would be very surprised if that were the case though. From ImportError: dlopen(/anaconda3/envs/fanova/lib/python3.6/site-packages/pyrfr/_regression.cpython-36m-darwin.so, 2): Symbol not found: __ZNKSt5ctypeIcE13_M_widen_initEv I'm guessing that something went wrong with swigging the pyrfr.

Could you only install pyrfr (through pip) in a new&clean conda environment and run

from pyrfr import regression

From that could you please send me the output that you get when installing pyrfr and (if an error occurs) also the trace again?

abhinavs95 commented 5 years ago

Does that mean that you can run the following minimal example?

Yes I can run the minimal example you posted above with no errors.

Could you only install pyrfr (through pip) in a new&clean conda environment and run

I could install pyrfr with no errors. Importing the regression package from pyrfr gives me the following (looks like the initial error I encountered):

>>> from pyrfr import regression
Traceback (most recent call last):
  File "/anaconda3/envs/f-test/lib/python3.6/site-packages/pyrfr/regression.py", line 14, in swig_import_helper
    return importlib.import_module(mname)
  File "/anaconda3/envs/f-test/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 658, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 571, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 922, in create_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: dlopen(/anaconda3/envs/f-test/lib/python3.6/site-packages/pyrfr/_regression.cpython-36m-darwin.so, 2): Symbol not found: __ZNKSt5ctypeIcE13_M_widen_initEv
  Referenced from: /anaconda3/envs/f-test/lib/python3.6/site-packages/pyrfr/_regression.cpython-36m-darwin.so
  Expected in: /usr/lib/libstdc++.6.dylib
 in /anaconda3/envs/f-test/lib/python3.6/site-packages/pyrfr/_regression.cpython-36m-darwin.so

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/anaconda3/envs/f-test/lib/python3.6/site-packages/pyrfr/regression.py", line 17, in <module>
    _regression = swig_import_helper()
  File "/anaconda3/envs/f-test/lib/python3.6/site-packages/pyrfr/regression.py", line 16, in swig_import_helper
    return importlib.import_module('_regression')
  File "/anaconda3/envs/f-test/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_regression'
AndreBiedenkapp commented 5 years ago

Okay that is very unfortunate. Some part of the pyrfr is not getting swigged correctly. It seems to me that it might be due to Mac OSX.

AndreBiedenkapp commented 5 years ago

Another potential way to fix the setup would be to (again) create a new&clean conda environment. In that environment please first run conda install gxx_linux-64 gcc_linux-64 swig. Then you can install pyrfr and fANOVA through pip.

Please report the result of running the two examples above.

abhinavs95 commented 5 years ago

@AndreBiedenkapp running conda install gxx_linux-64 gcc_linux-64 swig gives me the following:

Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - gxx_linux-64
  - gcc_linux-64

Current channels:

  - https://repo.anaconda.com/pkgs/main/osx-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/free/osx-64
  - https://repo.anaconda.com/pkgs/free/noarch
  - https://repo.anaconda.com/pkgs/r/osx-64
  - https://repo.anaconda.com/pkgs/r/noarch
  - https://repo.anaconda.com/pkgs/pro/osx-64
  - https://repo.anaconda.com/pkgs/pro/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

I think they are linux only packages and are not available for mac.

AndreBiedenkapp commented 5 years ago

That's surprising. Which Anaconda version are you using?

abhinavs95 commented 5 years ago

Conda version 4.5.11

AndreBiedenkapp commented 5 years ago

Anaconda 5.0 switched from OS-provided compiler tools to [their] own toolsets.

see conda-compiler-tools Looks like the packages I asked you to install are only available for anaconda 5.0 and higher. Could you please upgrade your conda to the latest version and try again?

abhinavs95 commented 5 years ago

@AndreBiedenkapp I have been trying to update conda to version 5. I tried a bunch of stuff including conda update conda and conda update anaconda. But after updating the version remains unchanged at 4.5.11. I'm not sure why this is happening.

The only other way is to remove current version and download version 5 fresh. But I'll be losing my current envs that way, so I'm looking for another fix.

AndreBiedenkapp commented 5 years ago

That's unfortunate. As I mentioned before, something with the swigging process of the pyrfr seems to be going wrong, which is most likely due to gcc version. If you don't intend on using Anaconda 5.0, it might be more difficult to setup everything correctly for mac.

Could you double check that the requirements listed here https://github.com/automl/random_forest_run are fullfilled?

abhinavs95 commented 5 years ago

Hi @AndreBiedenkapp yes I double checked the requirements. I installed fanova on a linux instance and things are working fine now. I will update on this thread if I am able to make it work on my mac (doing a fresh install using anaconda 5.0). Thank you for your help!