petsc4py not compatible with openblas

guyer commented 5 years ago

Issue: Installing petsc4py leads to blas_openblas variant dependencies, but then petsc4py won't import.

$ conda create --quiet --name test-environment-5 --show-channel-urls --channel conda-forge python=2.7 petsc4py
$ conda activate test-environment-5
$ python -c "from petsc4py import PETSc"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/guyer/anaconda/envs/test-environment-5/lib/python2.7/site-packages/petsc4py/PETSc.py", line 3, in <module>
    PETSc = ImportPETSc(ARCH)
  File "/Users/guyer/anaconda/envs/test-environment-5/lib/python2.7/site-packages/petsc4py/lib/__init__.py", line 29, in ImportPETSc
    return Import('petsc4py', 'PETSc', path, arch)
  File "/Users/guyer/anaconda/envs/test-environment-5/lib/python2.7/site-packages/petsc4py/lib/__init__.py", line 73, in Import
    module = import_module(pkg, name, path, arch)
  File "/Users/guyer/anaconda/envs/test-environment-5/lib/python2.7/site-packages/petsc4py/lib/__init__.py", line 58, in import_module
    with f: return imp.load_module(fullname, f, fn, info)
ImportError: dlopen(/Users/guyer/anaconda/envs/test-environment-5/lib/python2.7/site-packages/petsc4py/lib/PETSc.so, 2): Library not loaded: @rpath/libdmumps-5.1.2.dylib
  Referenced from: /Users/guyer/anaconda/envs/test-environment-5/lib/libpetsc.3.11.dylib
  Reason: image not found

Happens for both Python 2.7 and Python 3.6 on Travis-CI macOS image
Happens for both Python 2.7 and Python 3.6 on CircleCI linux image
passing blas=*=mkl resolves the problem
prior to updating conda from 4.6.14 to 4.7.5, mkl variants were installed in an environment created three days ago and everything worked fine

Environment (conda list):

``` $ conda list # packages in environment at /Users/guyer/anaconda/envs/test-environment-5: # # Name Version Build Channel blas 1.1 openblas conda-forge ca-certificates 2019.6.16 hecc5488_0 conda-forge certifi 2019.6.16 py27_0 conda-forge hypre 2.15.1 h69fd9fb_1000 conda-forge libblas 3.8.0 5_hd44dcd8_netlib conda-forge libcblas 3.8.0 5_hd44dcd8_netlib conda-forge libcxx 8.0.0 4 conda-forge libcxxabi 8.0.0 4 conda-forge libffi 3.2.1 h6de7cb9_1006 conda-forge libgfortran 3.0.1 0 conda-forge metis 5.1.0 h0a44026_1003 conda-forge mpi 1.0 mpich conda-forge mpich 3.2.1 ha90c164_1012 conda-forge mumps-mpi 5.1.2 blas_openblash9fe1a0a_1001 [blas_openblas] conda-forge ncurses 6.1 h0a44026_1002 conda-forge numpy 1.16.1 py27_blas_openblash486cb9f_0 [blas_openblas] conda-forge openblas 0.3.3 hdc02c5d_1001 conda-forge openssl 1.1.1b h01d97ff_2 conda-forge parmetis 4.0.3 hc4b10a8_1002 conda-forge petsc 3.11.1 hf985393_1 conda-forge petsc4py 3.11.0 py27hc745293_0 conda-forge pip 19.1.1 py27_0 conda-forge ptscotch 6.0.6 ha54644a_1002 conda-forge python 2.7.15 hd51d24c_1009 conda-forge readline 8.0 hcfe32e1_0 conda-forge scalapack 2.0.2 hd5f9c17_1004 conda-forge scotch 6.0.6 hde27766_1002 conda-forge setuptools 41.0.1 py27_0 conda-forge sqlite 3.28.0 hb7d70f7_1 conda-forge suitesparse 4.5.6 blas_openblash0bd06e0_1201 [blas_openblas] conda-forge tbb 2019.7 h770b8ee_0 conda-forge tk 8.6.9 h2573ce8_1002 conda-forge wheel 0.33.4 py27_0 conda-forge zlib 1.2.11 h1de35cc_1004 conda-forge ```

Details about conda and system ( conda info ):

``` $ conda info active environment : test-environment-5 active env location : /Users/guyer/anaconda/envs/test-environment-5 shell level : 6 user config file : /Users/guyer/.condarc populated config files : /Users/guyer/.condarc conda version : 4.7.5 conda-build version : 3.18.5 python version : 2.7.15.final.0 virtual packages : base environment : /Users/guyer/anaconda (writable) channel URLs : https://repo.anaconda.com/pkgs/main/osx-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/osx-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /Users/guyer/anaconda/pkgs /Users/guyer/.conda/pkgs envs directories : /Users/guyer/anaconda/envs /Users/guyer/.conda/envs platform : osx-64 user-agent : conda/4.7.5 requests/2.22.0 CPython/2.7.15 Darwin/18.6.0 OSX/10.14.5 UID:GID : 17723:17723 netrc file : None offline mode : False ```

dalcinl commented 5 years ago

This is maybe related to recent changes in conda and the removal of the free channel in anaconda.org, we have seen lots of breakages everywhere. I guess we just have to wait until we can mange to build a fresh MPI stack.

minrk commented 5 years ago

I don't think this has anything to do with free, it has to do with the splitting of the mumps package in 5.1.x. There was something mixed up in the mumps 5.1 builds, but the latest petsc was built with mumps 5.2.1, so conda install petsc4py works now:

$ conda create -y -n petsc4py petsc4py

``` Collecting package metadata (current_repodata.json): done Solving environment: done ## Package Plan ## environment location: /Users/benjaminrk/conda/envs/petsc4py added / updated specs: - petsc4py The following packages will be downloaded: package | build ---------------------------|----------------- mumps-include-5.2.1 | 0 22 KB conda-forge mumps-mpi-5.2.1 | haf446c3_0 3.0 MB conda-forge petsc-3.11.3 | hf985393_0 7.1 MB conda-forge ------------------------------------------------------------ Total: 10.1 MB The following NEW packages will be INSTALLED: bzip2 conda-forge/osx-64::bzip2-1.0.8-h01d97ff_0 ca-certificates conda-forge/osx-64::ca-certificates-2019.6.16-hecc5488_0 certifi conda-forge/osx-64::certifi-2019.6.16-py37_1 hypre conda-forge/osx-64::hypre-2.15.1-he0087a8_1001 libblas conda-forge/osx-64::libblas-3.8.0-11_openblas libcblas conda-forge/osx-64::libcblas-3.8.0-11_openblas libcxx conda-forge/osx-64::libcxx-8.0.0-4 libcxxabi conda-forge/osx-64::libcxxabi-8.0.0-4 libffi conda-forge/osx-64::libffi-3.2.1-h6de7cb9_1006 libgfortran conda-forge/osx-64::libgfortran-3.0.1-0 liblapack conda-forge/osx-64::liblapack-3.8.0-11_openblas libopenblas conda-forge/osx-64::libopenblas-0.3.6-hd44dcd8_6 metis conda-forge/osx-64::metis-5.1.0-h0a44026_1003 mpi conda-forge/osx-64::mpi-1.0-mpich mpich conda-forge/osx-64::mpich-3.2.1-ha90c164_1013 mumps-include conda-forge/osx-64::mumps-include-5.2.1-0 mumps-mpi conda-forge/osx-64::mumps-mpi-5.2.1-haf446c3_0 ncurses conda-forge/osx-64::ncurses-6.1-h0a44026_1002 numpy conda-forge/osx-64::numpy-1.17.0-py37h6b0580a_0 openssl conda-forge/osx-64::openssl-1.1.1c-h01d97ff_0 parmetis conda-forge/osx-64::parmetis-4.0.3-hc4b10a8_1002 petsc conda-forge/osx-64::petsc-3.11.3-hf985393_0 petsc4py conda-forge/osx-64::petsc4py-3.11.0-py37hc745293_0 pip conda-forge/osx-64::pip-19.2.1-py37_0 ptscotch conda-forge/osx-64::ptscotch-6.0.6-ha54644a_1002 python conda-forge/osx-64::python-3.7.3-h93065d6_1 readline conda-forge/osx-64::readline-8.0-hcfe32e1_0 scalapack conda-forge/osx-64::scalapack-2.0.2-h2db4217_1005 scotch conda-forge/osx-64::scotch-6.0.6-hde27766_1002 setuptools conda-forge/osx-64::setuptools-41.0.1-py37_0 sqlite conda-forge/osx-64::sqlite-3.29.0-hb7d70f7_0 suitesparse conda-forge/osx-64::suitesparse-4.5.6-h7019593_1202 tbb conda-forge/osx-64::tbb-2019.7-h770b8ee_0 tk conda-forge/osx-64::tk-8.6.9-h2573ce8_1002 wheel conda-forge/osx-64::wheel-0.33.4-py37_0 xz conda-forge/osx-64::xz-5.2.4-h1de35cc_1001 zlib conda-forge/osx-64::zlib-1.2.11-h01d97ff_1005 Downloading and Extracting Packages mumps-mpi-5.2.1 | 3.0 MB | ######################################################################################################################################################################################################################################################################################################################## | 100% petsc-3.11.3 | 7.1 MB | ######################################################################################################################################################################################################################################################################################################################## | 100% mumps-include-5.2.1 | 22 KB | ######################################################################################################################################################################################################################################################################################################################## | 100% Preparing transaction: done Verifying transaction: done Executing transaction: done # # To activate this environment, use # # $ conda activate petsc4py # # To deactivate an active environment, use # # $ conda deactivate ```

$ conda run -n petsc4py python -c "from petsc4py import PETSc; print('ok')"

conda-forge / petsc4py-feedstock

petsc4py not compatible with openblas #15