search

justinalsing / dlmmc

Dynamical linear modeling (DLM) regression code for analysis of atmospheric time-series data.
MIT License
23 stars 4 forks source link

installation failure with pip #3

Closed Chilipp closed 5 years ago

Chilipp commented 5 years ago

hey @justinalsing! As part of the software review in https://github.com/openjournals/joss-reviews/issues/1157 I report the following bug when compiling the stan models on my mac.

As outlined in the documentation, I installed all the dependencies through pip (I don't think that openmpi is available through pip)

conda create -n dlmmc python=3.7
conda activate dlmmc
pip install numpy scipy matplotlib netCDF4 pystan openmpi mpi4py
python compile_stan_models.py

The last command fails with

distutils.errors.CompileError: command '/usr/bin/gcc' failed with exit status 1 ``` DIAGNOSTIC(S) FROM PARSER: Warning: left-hand side variable (name=x_realization) occurs on right-hand side of assignment, causing inefficient deep copy to avoid aliasing. Warning: left-hand side variable (name=r) occurs on right-hand side of assignment, causing inefficient deep copy to avoid aliasing. Warning: left-hand side variable (name=r_realization) occurs on right-hand side of assignment, causing inefficient deep copy to avoid aliasing. Warning: left-hand side variable (name=M) occurs on right-hand side of assignment, causing inefficient deep copy to avoid aliasing. INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_386ef73bc045cd2ec5048e789ab86d0f NOW. Traceback (most recent call last): File "/Users/psommer/miniconda3/envs/dlmmc/lib/python3.7/distutils/unixccompiler.py", line 118, in _compile extra_postargs) File "/Users/psommer/miniconda3/envs/dlmmc/lib/python3.7/distutils/ccompiler.py", line 909, in spawn spawn(cmd, dry_run=self.dry_run) File "/Users/psommer/miniconda3/envs/dlmmc/lib/python3.7/distutils/spawn.py", line 36, in spawn _spawn_posix(cmd, search_path, dry_run=dry_run) File "/Users/psommer/miniconda3/envs/dlmmc/lib/python3.7/distutils/spawn.py", line 159, in _spawn_posix % (cmd, exit_status)) distutils.errors.DistutilsExecError: command '/usr/bin/gcc' failed with exit status 1 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "compile_stan_models.py", line 5, in model_vanilla_ar1 = pystan.StanModel(model_code=dlm_vanilla_ar1) File "/Users/psommer/miniconda3/envs/dlmmc/lib/python3.7/site-packages/pystan/model.py", line 349, in __init__ build_extension.run() File "/Users/psommer/miniconda3/envs/dlmmc/lib/python3.7/distutils/command/build_ext.py", line 339, in run self.build_extensions() File "/Users/psommer/miniconda3/envs/dlmmc/lib/python3.7/distutils/command/build_ext.py", line 448, in build_extensions self._build_extensions_serial() File "/Users/psommer/miniconda3/envs/dlmmc/lib/python3.7/distutils/command/build_ext.py", line 473, in _build_extensions_serial self.build_extension(ext) File "/Users/psommer/miniconda3/envs/dlmmc/lib/python3.7/distutils/command/build_ext.py", line 533, in build_extension depends=ext.depends) File "/Users/psommer/miniconda3/envs/dlmmc/lib/python3.7/distutils/ccompiler.py", line 574, in compile self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts) File "/Users/psommer/miniconda3/envs/dlmmc/lib/python3.7/distutils/unixccompiler.py", line 120, in _compile raise CompileError(msg) distutils.errors.CompileError: command '/usr/bin/gcc' failed with exit status 1 ```

here is the info on the environment:

conda info -a ``` active environment : dlmmc active env location : /Users/psommer/miniconda3/envs/dlmmc shell level : 2 user config file : /Users/psommer/.condarc populated config files : /Users/psommer/.condarc conda version : 4.5.12 conda-build version : 3.11.0 python version : 3.6.4.final.0 base environment : /Users/psommer/miniconda3 (writable) channel URLs : https://conda.anaconda.org/conda-forge/osx-64 https://conda.anaconda.org/conda-forge/noarch https://conda.anaconda.org/chilipp/label/dev/osx-64 https://conda.anaconda.org/chilipp/label/dev/noarch https://conda.anaconda.org/chilipp/label/master/osx-64 https://conda.anaconda.org/chilipp/label/master/noarch https://conda.anaconda.org/chilipp/osx-64 https://conda.anaconda.org/chilipp/noarch https://repo.anaconda.com/pkgs/main/osx-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/free/osx-64 https://repo.anaconda.com/pkgs/free/noarch https://repo.anaconda.com/pkgs/r/osx-64 https://repo.anaconda.com/pkgs/r/noarch https://repo.anaconda.com/pkgs/pro/osx-64 https://repo.anaconda.com/pkgs/pro/noarch package cache : /Users/psommer/miniconda3/pkgs /Users/psommer/.conda/pkgs envs directories : /Users/psommer/miniconda3/envs /Users/psommer/.conda/envs platform : osx-64 user-agent : conda/4.5.12 requests/2.18.4 CPython/3.6.4 Darwin/17.7.0 OSX/10.13.6 UID:GID : 502:20 netrc file : None offline mode : False # conda environments: # /Users/psommer/miniconda base /Users/psommer/miniconda3 dlmmc * /Users/psommer/miniconda3/envs/dlmmc sys.version: 3.6.4 |Anaconda, Inc.| (default, Jan 16 ... sys.prefix: /Users/psommer/miniconda3 sys.executable: /Users/psommer/miniconda3/bin/python conda location: /Users/psommer/miniconda3/lib/python3.6/site-packages/conda conda-build: /Users/psommer/miniconda3/bin/conda-build conda-convert: /Users/psommer/miniconda3/bin/conda-convert conda-develop: /Users/psommer/miniconda3/bin/conda-develop conda-env: /Users/psommer/miniconda3/bin/conda-env conda-index: /Users/psommer/miniconda3/bin/conda-index conda-inspect: /Users/psommer/miniconda3/bin/conda-inspect conda-metapackage: /Users/psommer/miniconda3/bin/conda-metapackage conda-render: /Users/psommer/miniconda3/bin/conda-render conda-server: /Users/psommer/miniconda3/bin/conda-server conda-skeleton: /Users/psommer/miniconda3/bin/conda-skeleton conda-smithy: /Users/psommer/miniconda3/bin/conda-smithy conda-verify: /Users/psommer/miniconda3/bin/conda-verify user site dirs: CIO_TEST: CONDA_DEFAULT_ENV: dlmmc CONDA_EXE: /Users/psommer/miniconda3/bin/conda CONDA_PREFIX: /Users/psommer/miniconda3/envs/dlmmc CONDA_PREFIX_1: /Users/psommer/miniconda3 CONDA_PROMPT_MODIFIER: (dlmmc) CONDA_PYTHON_EXE: /Users/psommer/miniconda3/bin/python CONDA_ROOT: /Users/psommer/miniconda3 CONDA_SHLVL: 2 PATH: /Users/psommer/miniconda3/envs/dlmmc/bin:/Users/psommer/miniconda3/bin:/usr/local/netcdf/bin:/Users/psommer/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin PYTHONWARNINGS: ignore:mode:DeprecationWarning:docutils.io:245 REQUESTS_CA_BUNDLE: SSL_CERT_FILE: ```
conda list ``` # packages in environment at /Users/psommer/miniconda3/envs/dlmmc: # # Name Version Build Channel appnope 0.1.0 backcall 0.1.0 bzip2 1.0.6 h1de35cc_1002 conda-forge ca-certificates 2018.11.29 ha4d7672_0 conda-forge certifi 2018.11.29 py37_1000 conda-forge cftime 1.0.3.4 cycler 0.10.0 Cython 0.29.2 decorator 4.3.0 ipykernel 5.1.0 ipython 7.2.0 ipython-genutils 0.2.0 jedi 0.13.2 jupyter-client 5.2.4 jupyter-core 4.4.0 kiwisolver 1.0.1 libcxx 7.0.0 h2d50403_2 conda-forge libffi 3.2.1 h0a44026_1005 conda-forge llvm-meta 7.0.0 0 conda-forge matplotlib 3.0.2 ncurses 6.1 h0a44026_1002 conda-forge netCDF4 1.4.2 numpy 1.16.0 openssl 1.0.2p h1de35cc_1002 conda-forge parso 0.3.1 pexpect 4.6.0 pickleshare 0.7.5 pip 18.1 py37_1000 conda-forge prompt-toolkit 2.0.7 ptyprocess 0.6.0 Pygments 2.3.1 pyparsing 2.3.1 pystan 2.18.1.0 python 3.7.1 h145921a_1000 conda-forge python-dateutil 2.7.5 pyzmq 17.1.2 readline 7.0 hcfe32e1_1001 conda-forge scipy 1.2.0 setuptools 40.6.3 py37_0 conda-forge six 1.12.0 sqlite 3.26.0 h1765d9f_1000 conda-forge tk 8.6.9 ha441bb4_1000 conda-forge tornado 5.1.1 traitlets 4.3.2 wcwidth 0.1.7 wheel 0.32.3 py37_0 conda-forge xz 5.2.4 h1de35cc_1001 conda-forge zlib 1.2.11 h1de35cc_1004 conda-forge ```

please note that it works without issues if I use the conda packages (which I personally would recommend anyway). In other words, the following works without any issues:

conda create -c conda-forge -n dlmmc2 python=3.7 numpy scipy matplotlib netCDF4 pystan ipykernel mpi4py -y
conda activate dlmmc2
python compile_stan_models.py

I strongly recommend to move away from pip in your installation instructions or to be more explicit in how to install the necessary dependencies.

Chilipp commented 5 years ago

hey @justinalsing, thanks for addressing the installation issues (https://github.com/openjournals/joss-reviews/issues/1157#issuecomment-459128351).

Note: If you make changes that address this issue in a commit or PR, please include the link to this issue (https://github.com/justinalsing/dlmmc/issues/3) in your commit message. It will be much easier to track and review the changes you implemented. Please also post a simple comment here when you are done with the suggested implementations (or if you have a different opinion).

justinalsing commented 5 years ago

Hi @Chilipp !

Thanks for this - I just edited the README to remove mention of pip as requested and added a note on which platforms the code runs on (Windows, Linux and Mac).

You're right there are some limitations to the functionality of pystan on Windows, but I have close collaborators who are using the code extensively on windows so these limitations don't appear to impact the use of the code. I believe the restrictions are isolated to the way pystan implements parallel processing on Windows, which is not integral to the functioning of the dlmmc code.

Thanks! Justin