search

conda-forge / numpy-feedstock

A conda-smithy repository for numpy.
BSD 3-Clause "New" or "Revised" License
8 stars 46 forks source link

Import error after upgrade of libopenblas #193

Closed Eleluku closed 4 years ago

Eleluku commented 4 years ago

Issue: As of today, importing jpye fails with the error ImportError: numpy.core.multiarray failed to import. When importing numpy directly, I get the error message below.

I found out, that the recent upgrade of the dependecy libopenblas from 0.3.9 to 0.3.10 broke this. After downgrading to 0.3.9 I was able to import numpy again.

Anyone else experiencing this behaviour?

import numpy as np

Error message:

Traceback (most recent call last):
  File "/home/eleluku/.conda/envs/myenv/lib/python3.7/site-packages/numpy/core/__init__.py", line 24, in <module>
    from . import multiarray
  File "/home/eleluku/.conda/envs/myenv/lib/python3.7/site-packages/numpy/core/multiarray.py", line 14, in <module>
    from . import overrides
  File "/home/eleluku/.conda/envs/myenv/lib/python3.7/site-packages/numpy/core/overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ImportError: libcblas.so.3: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/eleluku/example.py", line 3, in <module>
    import numpy
  File "/home/eleluku/.conda/envs/myenv/lib/python3.7/site-packages/numpy/__init__.py", line 142, in <module>
    from . import core
  File "/home/eleluku/.conda/envs/myenv/lib/python3.7/site-packages/numpy/core/__init__.py", line 50, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.7 from "/home/eleluku/.conda/envs/myenv/bin/python"
  * The NumPy version is: "1.18.5"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: libcblas.so.3: cannot open shared object file: No such file or directory

Environment (conda list):

$ conda list
    _libgcc_mutex:   0.1-conda_forge           conda-forge
    _openmp_mutex:   4.5-0_gnu                 conda-forge
    ca-certificates: 2020.4.5.2-hecda079_0     conda-forge
    certifi:         2020.4.5.2-py37hc8dfbb8_0 conda-forge
    jaydebeapi:      1.1.1-py_0                conda-forge
    jpype1:          0.6.3-py37h9de70de_1001   conda-forge
    libblas:         3.8.0-16_openblas         conda-forge
    libcblas:        3.8.0-16_openblas         conda-forge
    libedit:         3.1.20191231-h46ee950_0   conda-forge
    libffi:          3.2.1-he1b5a44_1007       conda-forge
    libgcc-ng:       9.2.0-h24d8f2e_2          conda-forge
    libgfortran-ng:  7.5.0-hdf63c60_6          conda-forge
    libgomp:         9.2.0-h24d8f2e_2          conda-forge
    liblapack:       3.8.0-16_openblas         conda-forge
    libopenblas:     0.3.10-h5ec1e0e_0          conda-forge
    libstdcxx-ng:    9.2.0-hdf63c60_2          conda-forge
    llvm-openmp:     10.0.0-hc9558a2_0         conda-forge
    ncurses:         6.1-hf484d3e_1002         conda-forge
    numpy:           1.18.5-py37h8960a57_0     conda-forge
    openssl:         1.1.1g-h516909a_0         conda-forge
    pip:             20.1.1-py_1               conda-forge
    python:          3.7.4-h265db76_1                     
    python_abi:      3.7-1_cp37m               conda-forge
    readline:        7.0-hf8c457e_1001         conda-forge
    setuptools:      47.3.0-py37hc8dfbb8_0     conda-forge
    sqlite:          3.31.1-h7b6447c_0                    
    tk:              8.6.10-hed695b0_0         conda-forge
    wheel:           0.34.2-py_1               conda-forge
    xz:              5.2.5-h516909a_0          conda-forge
    zlib:            1.2.11-h516909a_1006      conda-forge

Details about conda and system ( conda info ):

     active environment : None
            shell level : 0
       user config file : /home/eleluku/.condarc
 populated config files : 
          conda version : 4.4.10
    conda-build version : 3.4.1
         python version : 3.6.4.final.0
       base environment : /opt/anaconda3/anaconda3-5.1.0  (read only)
           channel URLs : https://repo.continuum.io/pkgs/main/linux-64
                          https://repo.continuum.io/pkgs/main/noarch
                          https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/linux-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
          package cache : /opt/anaconda3/anaconda3-5.1.0/pkgs
                          /home/eleluku/.conda/pkgs
       envs directories : /home/eleluku/.conda/envs
                          /opt/anaconda3/anaconda3-5.1.0/envs
               platform : linux-64
             user-agent : conda/4.4.10 requests/2.18.4 CPython/3.6.4 Linux/4.4.0-64-generic ubuntu/14.04 glibc/2.19
                UID:GID : 1204600235:1204600235
             netrc file : None
           offline mode : False

See also https://github.com/numpy/numpy/issues/16615

jakirkham commented 4 years ago

Are you able to reproduce this by just running import numpy without jpype?

Eleluku commented 4 years ago

Yes. The error traceback posted is the one I get when trying to do import numpy

When importing jpype I get the following error:

ImportError: numpy.core.multiarray failed to import
Traceback (most recent call last):
  File "/home/pew_admin/deployments/prod/pew/01_sources/whatson/pre_scripts/whatson/whatson_check_export_status.py", line 4, in <module>
    import jpype
  File "/home/pharos/.conda/envs/whatson-prescript/lib/python3.7/site-packages/jpype/__init__.py", line 17, in <module>
    from ._jpackage import *
  File "/home/pharos/.conda/envs/whatson-prescript/lib/python3.7/site-packages/jpype/_jpackage.py", line 18, in <module>
    import _jpype
ImportError: numpy.core.multiarray failed to import
isuruf commented 4 years ago

I see the following requirements of the libblas package,

     "libblas-3.8.0-16_openblas.tar.bz2": {
       "depends": [
         "libopenblas >=0.3.9,<0.3.10.0a0",
         "libopenblas >=0.3.9,<1.0a0"
       ],

which makes it impossible to install libblas=3.8.0=16_openblas and libopenblas=0.3.10.

I see that you've edited the conda info, to change libopenblas=0.3.9 to libopenblas=0.3.10.

Please don't open bogus issue reports.

Eleluku commented 4 years ago

Yeah, sorry for that edit. I realised that I copied the wrong conda info, with the working dependency. I should have created a new comment for that. This is the output of conda list for the failing version:

 _libgcc_mutex:   0.1-conda_forge           conda-forge
    _openmp_mutex:   4.5-0_gnu                 conda-forge
    ca-certificates: 2020.4.5.2-hecda079_0     conda-forge
    certifi:         2020.4.5.2-py37hc8dfbb8_0 conda-forge
    jaydebeapi:      1.1.1-py_0                conda-forge
    jpype1:          0.6.3-py37h9de70de_1001   conda-forge
    libblas:         3.8.0-16_openblas         conda-forge
    libcblas:        3.8.0-16_openblas         conda-forge
    libedit:         3.1.20191231-h46ee950_0   conda-forge
    libffi:          3.2.1-he1b5a44_1007       conda-forge
    libgcc-ng:       9.2.0-h24d8f2e_2          conda-forge
    libgfortran-ng:  7.5.0-hdf63c60_6          conda-forge
    libgomp:         9.2.0-h24d8f2e_2          conda-forge
    liblapack:       3.8.0-16_openblas         conda-forge
    libopenblas:     0.3.10-h5ec1e0e_0         conda-forge
    libstdcxx-ng:    9.2.0-hdf63c60_2          conda-forge
    llvm-openmp:     10.0.0-hc9558a2_0         conda-forge
    ncurses:         6.1-hf484d3e_1002         conda-forge
    numpy:           1.18.5-py37h8960a57_0     conda-forge
    openssl:         1.1.1g-h516909a_0         conda-forge
    pip:             20.1.1-py_1               conda-forge
    python:          3.7.4-h265db76_1                     
    python_abi:      3.7-1_cp37m               conda-forge
    readline:        7.0-hf8c457e_1001         conda-forge
    setuptools:      47.3.0-py37hc8dfbb8_0     conda-forge
    sqlite:          3.31.1-h7b6447c_0                    
    tk:              8.6.10-hed695b0_0         conda-forge
    wheel:           0.34.2-py_1               conda-forge
    xz:              5.2.5-h516909a_0          conda-forge
    zlib:            1.2.11-h516909a_1006      conda-forge
Eleluku commented 4 years ago

The conda env was created like this:

conda create -n myenv -c conda-forge python=3.7.4 jaydebeapi jpype1=0.6.3

isuruf commented 4 years ago

Here is what I get when I do that,

  _libgcc_mutex      conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge
  _openmp_mutex      conda-forge/linux-64::_openmp_mutex-4.5-1_llvm
  ca-certificates    conda-forge/linux-64::ca-certificates-2020.4.5.2-hecda079_0
  certifi            conda-forge/linux-64::certifi-2020.4.5.2-py37hc8dfbb8_0
  jaydebeapi         conda-forge/noarch::jaydebeapi-1.1.1-py_0
  jpype1             conda-forge/linux-64::jpype1-0.6.3-py37h9de70de_1001
  libblas            conda-forge/linux-64::libblas-3.8.0-16_openblas
  libcblas           conda-forge/linux-64::libcblas-3.8.0-16_openblas
  libedit            conda-forge/linux-64::libedit-3.1.20191231-h46ee950_0
  libffi             conda-forge/linux-64::libffi-3.2.1-he1b5a44_1007
  libgcc-ng          conda-forge/linux-64::libgcc-ng-9.2.0-h24d8f2e_2
  libgfortran-ng     conda-forge/linux-64::libgfortran-ng-7.5.0-hdf63c60_6
  liblapack          conda-forge/linux-64::liblapack-3.8.0-16_openblas
  libopenblas        conda-forge/linux-64::libopenblas-0.3.9-h5ec1e0e_0
  libstdcxx-ng       conda-forge/linux-64::libstdcxx-ng-9.2.0-hdf63c60_2
  llvm-openmp        conda-forge/linux-64::llvm-openmp-10.0.0-hc9558a2_0
  ncurses            conda-forge/linux-64::ncurses-6.1-hf484d3e_1002
  numpy              conda-forge/linux-64::numpy-1.18.5-py37h8960a57_0
  openssl            conda-forge/linux-64::openssl-1.1.1g-h516909a_0
  pip                conda-forge/noarch::pip-20.1.1-py_1
  python             pkgs/main/linux-64::python-3.7.4-h265db76_1
  python_abi         conda-forge/linux-64::python_abi-3.7-1_cp37m
  readline           conda-forge/linux-64::readline-7.0-hf8c457e_1001
  setuptools         conda-forge/linux-64::setuptools-47.3.0-py37hc8dfbb8_0
  sqlite             pkgs/main/linux-64::sqlite-3.31.1-h7b6447c_0
  tk                 conda-forge/linux-64::tk-8.6.10-hed695b0_0
  wheel              conda-forge/noarch::wheel-0.34.2-py_1
  xz                 conda-forge/linux-64::xz-5.2.5-h516909a_0
  zlib               conda-forge/linux-64::zlib-1.2.11-h516909a_1006
Eleluku commented 4 years ago

I just tried it on different servers with different conda versions. It may be because of the old version of conda we are using. When creating the environment with conda 4.7.12 I get the same result as you. When creating with conda 4.4.10, it uses the 0.3.10 version of libopenblas.

So it seems this is resolved by updating conda to a newer version.