cavalab / feat

A feature engineering automation tool for learning data representations
https://cavalab.org/feat
GNU General Public License v3.0
33 stars 14 forks source link

Segmentation fault #264

Closed chloeh13q closed 4 years ago

chloeh13q commented 4 years ago

Hi!

I went through quite an ordeal to try to install FEAT, and finally succeded. However, when I tried to test the package with ./build/feat docs/examples/data/d_enc.csv -rs 42 -g 2 -p 5, I ran into a segmentation fault. I also wasn't able to import the package into Jupyter notebook. Details are as follows:

- System: macOS Catalina

- Conda environment: I built a conda environment from scratch for FEAT so it should only contain packages required by it.

# Name                    Version                   Build  Channel
appnope                   0.1.0                    pypi_0    pypi
argon2-cffi               20.1.0                   pypi_0    pypi
arpack                    3.7.0                hbb27dc6_1    conda-forge
attrs                     20.1.0                   pypi_0    pypi
backcall                  0.2.0                    pypi_0    pypi
bleach                    3.1.5                    pypi_0    pypi
bzip2                     1.0.8                haf1e3a3_3    conda-forge
c-ares                    1.16.1               haf1e3a3_3    conda-forge
ca-certificates           2020.6.20            hecda079_0    conda-forge
certifi                   2020.6.20        py36h9f0ad1d_0    conda-forge
cffi                      1.14.2                   pypi_0    pypi
click                     7.1.2                    pypi_0    pypi
ctags                     5.8               h1de35cc_1000    conda-forge
cython                    0.29.21          py36hb1e8313_0
decorator                 4.4.2                    pypi_0    pypi
defusedxml                0.6.0                    pypi_0    pypi
eigency                   1.77                     pypi_0    pypi
entrypoints               0.3                      pypi_0    pypi
future                    0.18.2                   pypi_0    pypi
glpk                      4.65              h65ac59c_1002    conda-forge
gmp                       6.2.0                h4a8c4bd_2    conda-forge
hdf5                      1.10.6          nompi_h3e39495_100    conda-forge
icu                       67.1                 h4a8c4bd_0    conda-forge
importlib-metadata        1.7.0                    pypi_0    pypi
ipykernel                 5.3.4                    pypi_0    pypi
ipython                   7.16.1                   pypi_0    pypi
ipython-genutils          0.2.0                    pypi_0    pypi
jedi                      0.17.2                   pypi_0    pypi
jinja2                    2.11.2                   pypi_0    pypi
joblib                    0.16.0                   pypi_0    pypi
json-c                    0.12.1               h6b7fe83_2
jsonschema                3.2.0                    pypi_0    pypi
jupyter-client            6.1.7                    pypi_0    pypi
jupyter-core              4.6.3                    pypi_0    pypi
krb5                      1.17.1               h14dd6a4_2    conda-forge
libblas                   3.8.0               17_openblas    conda-forge
libcblas                  3.8.0               17_openblas    conda-forge
libcurl                   7.71.1               h9bf37e3_5    conda-forge
libcxx                    10.0.0                        1
libedit                   3.1.20191231         h1de35cc_1
libev                     4.33                 haf1e3a3_0    conda-forge
libffi                    3.3                  hb1e8313_2
libgfortran               4.0.0                         2    conda-forge
libiconv                  1.16                 haf1e3a3_0    conda-forge
liblapack                 3.8.0               17_openblas    conda-forge
liblapacke                3.8.0               17_openblas    conda-forge
libnghttp2                1.41.0               h8a08a2b_1    conda-forge
libopenblas               0.3.10          openmp_h63d9170_4    conda-forge
libprotobuf               3.11.4               hd174df1_0    conda-forge
libssh2                   1.9.0                h8a08a2b_5    conda-forge
libxml2                   2.9.10               h2c6e4a5_2    conda-forge
livereload                2.6.3                    pypi_0    pypi
llvm-openmp               10.0.1               h28b9765_0    conda-forge
lunr                      0.5.6                    pypi_0    pypi
lzo                       2.10              haf1e3a3_1000    conda-forge
markdown                  3.2.2                    pypi_0    pypi
markupsafe                1.1.1                    pypi_0    pypi
mistune                   0.8.4                    pypi_0    pypi
mkdocs                    1.1                      pypi_0    pypi
mkdocs-material           5.5.11                   pypi_0    pypi
mkdocs-material-extensions 1.0                      pypi_0    pypi
nbconvert                 5.6.1                    pypi_0    pypi
nbformat                  5.0.7                    pypi_0    pypi
ncurses                   6.2                  h0a44026_1
nltk                      3.5                      pypi_0    pypi
notebook                  6.1.3                    pypi_0    pypi
numpy                     1.19.1                   pypi_0    pypi
openssl                   1.1.1g               haf1e3a3_1    conda-forge
packaging                 20.4                     pypi_0    pypi
pandas                    1.1.1                    pypi_0    pypi
pandocfilters             1.4.2                    pypi_0    pypi
parso                     0.7.1                    pypi_0    pypi
pexpect                   4.8.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pip                       20.2.2                   py36_0
prometheus-client         0.8.0                    pypi_0    pypi
prompt-toolkit            3.0.6                    pypi_0    pypi
ptyprocess                0.6.0                    pypi_0    pypi
pycparser                 2.20                     pypi_0    pypi
pygments                  2.6.1                    pypi_0    pypi
pymdown-extensions        8.0                      pypi_0    pypi
pyparsing                 2.4.7                    pypi_0    pypi
pyrsistent                0.16.0                   pypi_0    pypi
python                    3.6.10               hf48f09d_2
python-dateutil           2.8.1                    pypi_0    pypi
python_abi                3.6                     1_cp36m    conda-forge
pytz                      2020.1                   pypi_0    pypi
pyyaml                    5.3.1                    pypi_0    pypi
pyzmq                     19.0.2                   pypi_0    pypi
readline                  8.0                  h1de35cc_0
regex                     2020.7.14                pypi_0    pypi
scikit-learn              0.23.2                   pypi_0    pypi
scipy                     1.5.2                    pypi_0    pypi
send2trash                1.5.0                    pypi_0    pypi
setuptools                49.6.0                   pypi_0    pypi
shogun-cpp                6.1.4                h5397d2f_3    conda-forge
six                       1.15.0                   pypi_0    pypi
snappy                    1.1.8                h4a8c4bd_3    conda-forge
sqlite                    3.33.0               hffcf06c_0
terminado                 0.8.3                    pypi_0    pypi
testpath                  0.4.4                    pypi_0    pypi
threadpoolctl             2.1.0                    pypi_0    pypi
tk                        8.6.10               hb0a8c7a_0
tornado                   6.0.4                    pypi_0    pypi
tqdm                      4.48.2                   pypi_0    pypi
traitlets                 4.3.3                    pypi_0    pypi
wcwidth                   0.2.5                    pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
wheel                     0.35.1                     py_0
xz                        5.2.5                h1de35cc_0
zipp                      3.1.0                    pypi_0    pypi
zlib                      1.2.11               h1de35cc_3

- Terminal error message:

load_csv...dtypes: x0 (f), x1 (f), x2 (f), x3 (f), x4 (f), x5 (c), x6 (f), x7 (c),
fitting model...
/////////////////////////////////////////////////////////////////////
//           * Feature Engineering Automation Tool *               //
// La Cava et al. 2017                                             //
// License: GPL v3                                                 //
/////////////////////////////////////////////////////////////////////

X data types: f, f, f, f, f, c, f, c,
functions set: [+, -, *, /, ^2, cube, sqrt, sin, cos, exp, log, ^, logit, tanh, gaussian, relu, split, split_c, fuzzy_split, fuzzy_split_c, fuzzy_fixed_split, fuzzy_fixed_split_c, b2f, c2f, and, or, not, xor, =, <, <=, >, >=, if, ite, ]

scorer: mse
Setting up data
Fitting initial model
initial model: [x0][x1][x2][x3][x4][x5][x6][x7]
[1]    3560 segmentation fault  ./build/feat docs/examples/data/d_enc.csv -rs 42 -g 2 -p 5

- Jupyter notebook error message: When I do "from feat import Feat", I get the following in terminal:

python(3826,0x1135a8dc0) malloc: *** error for object 0x12cbc9ac0: pointer being freed was not allocated
python(3826,0x1135a8dc0) malloc: *** set a breakpoint in malloc_error_break to debug
folivetti commented 4 years ago

see the issue I opened some days ago https://github.com/lacava/feat/issues/263

There is a featenv.yml file at the very bottom that I used to successfully create a proper conda environment to build FEAT. Maybe it can help you as well.

chloeh13q commented 4 years ago

@folivetti Thanks for your reply. I tried your conda environment setup, but am getting a bunch of ResolvePackageNotFound errors, which I think might be due to platform-specific build constraints. Manually installing these doesn't seem to work either, as I'm getting package conflicts in conda. Could you do a conda env export with the --no-build flag?

folivetti commented 4 years ago

@chloeh13q

of course! Here it goes!

nobuild-env.yml.txt

lacava commented 4 years ago

@folivetti thanks for sharing the conda environment from #263 !

@chloeh13q I haven't built FEAT on a mac, but I imagine getting the conda environment right would work. I'm working on providing a minimal conda env in the repo. At some point this summer an updated conda install began causing a package conflict for FEAT, probably in relation to shogun. the miniconda install we use for testing (Miniconda3-4.7.12.1) still seems to work ok, as well as the environment @folivetti provided.

Let me know if the environment setup fixes the issue.

chloeh13q commented 4 years ago

@folivetti @lacava Hi! Thank you both for your help and the info! I think it might in fact have to do with the fact that I'm using a Mac - I tried to create a new conda environment using @folivetti 's yml configurations, but keep running into package conflicts, after which conda would just run for hours trying to resolve these conflicts, and I've had to kill it because it was taking way too long. I think other Mac users have also run into this problem where conda is reporting weird conflicts when theoretically there should be none. @lacava I followed your .travis_install.sh script and installed all of the relevant packages (apart from the fact that I use Anaconda instead of miniconda). I also checked against @folivetti 's yml file to make sure that I have the same versions. But no luck so far. Just a side note, not sure if this might have to do with the issue: I actually struggled with running the ./install script, and had to manually add the path of shogun library into link.txt in order to run it. Anyways, I'll keep trying!

chloeh13q commented 4 years ago

@folivetti @lacava Update: With @folivetti 's environment yml, I successfully installed the package on Ubuntu (Ubuntu 20.04.1)! Although I did run into a couple of problems here and there in the installation, it was much, much smoother than the macOS installation. I think the issue might just be due to OS and conda imcompatibilities. I don't know if you want to look into macOS installations in the future, but I'll go ahead and close this issue for now. Once again, thank you both for your help!

lacava commented 4 years ago

glad to hear it @chloeh13q! if there are any specific problems you ran into with Ubuntu 20, let me know. I know the eigen source needs to be updated (#265) and the environment specified, but there may be other issues I'm not aware of. I'll open up a separate issue for addressing macOS install.

lacava commented 4 years ago

@chloeh13q for your reference, a conda environment template is now available here.