search

pymc-devs / pymc

Bayesian Modeling and Probabilistic Programming in Python
https://docs.pymc.io/
Other
8.71k stars 2.01k forks source link

Conda can select old PyMC version with "conda install -c conda-forge pymc" #5875

Closed maresb closed 1 year ago

maresb commented 2 years ago

Description of your problem

Please provide a minimal, self-contained, and reproducible example.

On Linux,

conda create -y --name pymc4-5875 -c conda-forge python=3.9
conda activate pymc4-5875
conda install -y -c conda-forge pymc

results in the installation of conda-forge/linux-64::pymc-4.0.0b6-py39ha563b64_0. This beta version doesn't install mkl-service and results in

$ python -c "import pymc"
/tmp/tmp55mck6a7/source.c:1:10: fatal error: cblas.h: No such file or directory
    1 | #include <cblas.h>
      |          ^~~~~~~~~
compilation terminated.

It's reported that subsequent inference is very slow.

Please provide the full traceback.

``` $ conda create -y --name pymc4-5875 -c conda-forge python=3.9 Collecting package metadata (current_repodata.json): done Solving environment: done ==> WARNING: A newer version of conda exists. <== current version: 4.11.0 latest version: 4.13.0 Please update conda by running $ conda update -n base conda ## Package Plan ## environment location: ~/micromamba/envs/pymc4-5875 added / updated specs: - python=3.9 The following NEW packages will be INSTALLED: _libgcc_mutex conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge _openmp_mutex conda-forge/linux-64::_openmp_mutex-4.5-2_gnu bzip2 conda-forge/linux-64::bzip2-1.0.8-h7f98852_4 ca-certificates conda-forge/linux-64::ca-certificates-2022.5.18.1-ha878542_0 ld_impl_linux-64 conda-forge/linux-64::ld_impl_linux-64-2.36.1-hea4e1c9_2 libffi conda-forge/linux-64::libffi-3.4.2-h7f98852_5 libgcc-ng conda-forge/linux-64::libgcc-ng-12.1.0-h8d9b700_16 libgomp conda-forge/linux-64::libgomp-12.1.0-h8d9b700_16 libnsl conda-forge/linux-64::libnsl-2.0.0-h7f98852_0 libuuid conda-forge/linux-64::libuuid-2.32.1-h7f98852_1000 libzlib conda-forge/linux-64::libzlib-1.2.12-h166bdaf_0 ncurses conda-forge/linux-64::ncurses-6.3-h27087fc_1 openssl conda-forge/linux-64::openssl-3.0.3-h166bdaf_0 pip conda-forge/noarch::pip-22.1.2-pyhd8ed1ab_0 python conda-forge/linux-64::python-3.9.13-h2660328_0_cpython python_abi conda-forge/linux-64::python_abi-3.9-2_cp39 readline conda-forge/linux-64::readline-8.1-h46c0cb4_0 setuptools conda-forge/linux-64::setuptools-62.3.3-py39hf3d152e_0 sqlite conda-forge/linux-64::sqlite-3.38.5-h4ff8645_0 tk conda-forge/linux-64::tk-8.6.12-h27826a3_0 tzdata conda-forge/noarch::tzdata-2022a-h191b570_0 wheel conda-forge/noarch::wheel-0.37.1-pyhd8ed1ab_0 xz conda-forge/linux-64::xz-5.2.5-h516909a_1 zlib conda-forge/linux-64::zlib-1.2.12-h166bdaf_0 Preparing transaction: done Verifying transaction: done Executing transaction: done # # To activate this environment, use # # $ conda activate pymc4-5875 # # To deactivate an active environment, use # # $ conda deactivate $ conda activate pymc4-5875 $ conda install -y -c conda-forge pymc Collecting package metadata (current_repodata.json): done Solving environment: failed with initial frozen solve. Retrying with flexible solve. Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): done Solving environment: done ==> WARNING: A newer version of conda exists. <== current version: 4.11.0 latest version: 4.13.0 Please update conda by running $ conda update -n base conda ## Package Plan ## environment location: ~/micromamba/envs/pymc4-5875 added / updated specs: - pymc The following NEW packages will be INSTALLED: aeppl conda-forge/noarch::aeppl-0.0.27-pyhd8ed1ab_0 aesara conda-forge/linux-64::aesara-2.5.1-py39h788985e_0 arviz conda-forge/noarch::arviz-0.12.1-pyhd8ed1ab_0 binutils conda-forge/linux-64::binutils-2.36.1-hdd6e379_2 binutils_impl_lin~ conda-forge/linux-64::binutils_impl_linux-64-2.36.1-h193b22a_2 binutils_linux-64 conda-forge/linux-64::binutils_linux-64-2.36-hf3e587d_10 brotli conda-forge/linux-64::brotli-1.0.9-h166bdaf_7 brotli-bin conda-forge/linux-64::brotli-bin-1.0.9-h166bdaf_7 c-ares conda-forge/linux-64::c-ares-1.18.1-h7f98852_0 c-compiler conda-forge/linux-64::c-compiler-1.4.2-h166bdaf_0 cachetools conda-forge/noarch::cachetools-5.0.0-pyhd8ed1ab_0 certifi conda-forge/linux-64::certifi-2022.5.18.1-py39hf3d152e_0 cftime conda-forge/linux-64::cftime-1.6.0-py39hd257fcd_1 cloudpickle conda-forge/noarch::cloudpickle-2.1.0-pyhd8ed1ab_0 cons conda-forge/noarch::cons-0.4.5-pyhd8ed1ab_0 curl conda-forge/linux-64::curl-7.83.1-h2283fc2_0 cxx-compiler conda-forge/linux-64::cxx-compiler-1.4.2-h924138e_0 cycler conda-forge/noarch::cycler-0.11.0-pyhd8ed1ab_0 etuples conda-forge/noarch::etuples-0.3.5-pyhd8ed1ab_0 fastprogress conda-forge/noarch::fastprogress-1.0.2-pyhd8ed1ab_0 filelock conda-forge/noarch::filelock-3.7.1-pyhd8ed1ab_0 fonttools conda-forge/linux-64::fonttools-4.33.3-py39hb9d737c_0 freetype conda-forge/linux-64::freetype-2.10.4-h0708190_1 gcc conda-forge/linux-64::gcc-10.3.0-he2824d0_10 gcc_impl_linux-64 conda-forge/linux-64::gcc_impl_linux-64-10.3.0-hf2f2afa_16 gcc_linux-64 conda-forge/linux-64::gcc_linux-64-10.3.0-hc39de41_10 giflib conda-forge/linux-64::giflib-5.2.1-h36c2ea0_2 gxx conda-forge/linux-64::gxx-10.3.0-he2824d0_10 gxx_impl_linux-64 conda-forge/linux-64::gxx_impl_linux-64-10.3.0-hf2f2afa_16 gxx_linux-64 conda-forge/linux-64::gxx_linux-64-10.3.0-h2593f52_10 hdf4 conda-forge/linux-64::hdf4-4.2.15-h10796ff_3 hdf5 conda-forge/linux-64::hdf5-1.12.1-nompi_h4df4325_104 importlib-metadata conda-forge/linux-64::importlib-metadata-4.11.4-py39hf3d152e_0 importlib_metadata conda-forge/noarch::importlib_metadata-4.11.4-hd8ed1ab_0 jpeg conda-forge/linux-64::jpeg-9e-h166bdaf_1 kernel-headers_li~ conda-forge/noarch::kernel-headers_linux-64-2.6.32-he073ed8_15 keyutils conda-forge/linux-64::keyutils-1.6.1-h166bdaf_0 kiwisolver conda-forge/linux-64::kiwisolver-1.4.2-py39hf939315_1 krb5 conda-forge/linux-64::krb5-1.19.3-h08a2579_0 lcms2 conda-forge/linux-64::lcms2-2.12-hddcbb42_0 lerc conda-forge/linux-64::lerc-3.0-h9c3ff4c_0 libblas conda-forge/linux-64::libblas-3.9.0-15_linux64_openblas libbrotlicommon conda-forge/linux-64::libbrotlicommon-1.0.9-h166bdaf_7 libbrotlidec conda-forge/linux-64::libbrotlidec-1.0.9-h166bdaf_7 libbrotlienc conda-forge/linux-64::libbrotlienc-1.0.9-h166bdaf_7 libcblas conda-forge/linux-64::libcblas-3.9.0-15_linux64_openblas libcurl conda-forge/linux-64::libcurl-7.83.1-h2283fc2_0 libdeflate conda-forge/linux-64::libdeflate-1.10-h7f98852_0 libedit conda-forge/linux-64::libedit-3.1.20191231-he28a2e2_2 libev conda-forge/linux-64::libev-4.33-h516909a_1 libgcc-devel_linu~ conda-forge/linux-64::libgcc-devel_linux-64-10.3.0-he6cfe16_16 libgfortran-ng conda-forge/linux-64::libgfortran-ng-12.1.0-h69a702a_16 libgfortran5 conda-forge/linux-64::libgfortran5-12.1.0-hdcd56e2_16 liblapack conda-forge/linux-64::liblapack-3.9.0-15_linux64_openblas libnetcdf conda-forge/linux-64::libnetcdf-4.8.1-nompi_h329d8a1_102 libnghttp2 conda-forge/linux-64::libnghttp2-1.47.0-he49606f_0 libopenblas conda-forge/linux-64::libopenblas-0.3.20-pthreads_h78a6416_0 libpng conda-forge/linux-64::libpng-1.6.37-h21135ba_2 libsanitizer conda-forge/linux-64::libsanitizer-10.3.0-h26c7422_16 libssh2 conda-forge/linux-64::libssh2-1.10.0-ha35d2d1_2 libstdcxx-devel_l~ conda-forge/linux-64::libstdcxx-devel_linux-64-10.3.0-he6cfe16_16 libstdcxx-ng conda-forge/linux-64::libstdcxx-ng-12.1.0-ha89aaad_16 libtiff conda-forge/linux-64::libtiff-4.4.0-h0fcbabc_0 libwebp conda-forge/linux-64::libwebp-1.2.2-h3452ae3_0 libwebp-base conda-forge/linux-64::libwebp-base-1.2.2-h7f98852_1 libxcb conda-forge/linux-64::libxcb-1.13-h7f98852_1004 libzip conda-forge/linux-64::libzip-1.8.0-h1c5bbd1_1 logical-unificati~ conda-forge/noarch::logical-unification-0.4.5-pyhd8ed1ab_0 lz4-c conda-forge/linux-64::lz4-c-1.9.3-h9c3ff4c_1 matplotlib-base conda-forge/linux-64::matplotlib-base-3.5.2-py39h700656a_0 minikanren conda-forge/noarch::minikanren-1.0.3-pyhd8ed1ab_0 multipledispatch conda-forge/noarch::multipledispatch-0.6.0-py_0 munkres conda-forge/noarch::munkres-1.1.4-pyh9f0ad1d_0 netcdf4 conda-forge/linux-64::netcdf4-1.5.8-nompi_py39h64b754b_101 numpy conda-forge/linux-64::numpy-1.22.4-py39hc58783e_0 openjpeg conda-forge/linux-64::openjpeg-2.4.0-hb52868f_1 packaging conda-forge/noarch::packaging-21.3-pyhd8ed1ab_0 pandas conda-forge/linux-64::pandas-1.4.2-py39h1832856_2 pillow conda-forge/linux-64::pillow-9.1.1-py39hae2aec6_1 pthread-stubs conda-forge/linux-64::pthread-stubs-0.4-h36c2ea0_1001 pymc conda-forge/linux-64::pymc-4.0.0b6-py39ha563b64_0 pyparsing conda-forge/noarch::pyparsing-3.0.9-pyhd8ed1ab_0 python-dateutil conda-forge/noarch::python-dateutil-2.8.2-pyhd8ed1ab_0 pytz conda-forge/noarch::pytz-2022.1-pyhd8ed1ab_0 scipy conda-forge/linux-64::scipy-1.8.1-py39he49c0e8_0 semver conda-forge/noarch::semver-2.13.0-pyh9f0ad1d_0 six conda-forge/noarch::six-1.16.0-pyh6c4a22f_0 sysroot_linux-64 conda-forge/noarch::sysroot_linux-64-2.12-he073ed8_15 toolz conda-forge/noarch::toolz-0.11.2-pyhd8ed1ab_0 typing-extensions conda-forge/noarch::typing-extensions-3.10.0.2-hd8ed1ab_0 typing_extensions conda-forge/noarch::typing_extensions-3.10.0.2-pyha770c72_0 unicodedata2 conda-forge/linux-64::unicodedata2-14.0.0-py39hb9d737c_1 xarray conda-forge/noarch::xarray-2022.3.0-pyhd8ed1ab_0 xarray-einstats conda-forge/noarch::xarray-einstats-0.2.2-pyhd8ed1ab_0 xorg-libxau conda-forge/linux-64::xorg-libxau-1.0.9-h7f98852_0 xorg-libxdmcp conda-forge/linux-64::xorg-libxdmcp-1.1.3-h7f98852_0 zipp conda-forge/noarch::zipp-3.8.0-pyhd8ed1ab_0 zstd conda-forge/linux-64::zstd-1.5.2-h8a70e8d_1 Preparing transaction: done Verifying transaction: done Executing transaction: done ```

Please provide any additional information below.

This was originally reported in https://github.com/conda-forge/pymc-feedstock/pull/41#issuecomment-1148135445 and https://github.com/conda-forge/pymc-feedstock/issues/43 but with slightly different conditions.

I have found two independent ways to resolve the issue:

  1. In the conda install command, indicate the version with "pymc>=4". A few caveats here: neither pymc=4 nor pymc=4.0.0 exclude 4.0.0b6. The quotation marks are mandatory to prevent Bash from interpreting the > character as an output redirection to a file named =4.
  2. Use Mamba instead of Conda.

I wrote some additional thoughts on this issue in https://github.com/conda-forge/pymc-feedstock/issues/43#issuecomment-1150957693.

I wonder if the installation instructions should be updated to recommend either or both of:

twiecki commented 2 years ago

I updated the wiki installation instructions for now.

maresb commented 2 years ago

Excellent! That should solve this issue (I hope).

While we're thinking about it, it might be clever to add a "Fancy Install" section to Linux and Windows.

Suggestion for the Windows instructions:

...for those who would prefer to manage the GCC compiler installation on their own...

→

...for those who would prefer to manage the BLAS and GCC compiler installations on their own...

And then removing mention of Chocolatey (and also numba?) we could copy "Fancy Install" to Linux/Mac.

We should probably mention in the case of M1 that we use OpenBLAS. (helpful link to the exact Conda package names)

winedarksea commented 2 years ago

Seeing this issue as well, downloading the beta release by default. And following the updated wiki instructions (>=4) I get the following error:

Encountered problems while solving:
  - nothing provides vc 9.* needed by mkl-service-2.0.1-py27h0c8e037_0

ran it a second time and got something slightly different

Encountered problems while solving:
  - nothing provides mkl >=2019.3,<2020.0a0 needed by mkl-service-2.0.1-py27h0c8e037_0

this was with mamba, not conda directly (Windows, AMD64) interestingly this seems to be from using OpenBlas (nomkl)/mamba and using conda directly and allowing mkl looks like it downloads correctly.

cluhmann commented 1 year ago

Closing because the installation instructions seem to have taken care of the primary issue. Also potentially surpassed by https://github.com/pymc-devs/pymc/issues/6750.