conda-forge / openblas-feedstock

A conda-smithy repository for openblas.
BSD 3-Clause "New" or "Revised" License
9 stars 38 forks source link

OpenBLAS Numpy's dot product hangs if OpenCV is imported #96

Closed guillaumedavidphd closed 4 years ago

guillaumedavidphd commented 4 years ago

Issue:

Numpy's dot product hangs forever if OpenCV has been imported as well. It sometimes happens on the first dot product execution but not always. The code example below therefore loops on the dot product an arbitrary number of times. nump.linalg.eig first because it seems this bug has something to do with the linear algebra libraries. This happens on Ubuntu 19.10 if the default OpenBLAS libraries are used. Switching to the MKL libraries solves the problem. The bug also disappears if smaller matrices are used. Memory isn't freed at each iteration, so when memory usage accumulates about ~4 GB, it crashes.

Reproducing code example:

conda create -n npcrash
conda install numpy opencv -c conda-forge -n npcrash
conda activate npcrash
python npcrash.py

Python script npcrash.py:

import numpy as np
import cv2

S = np.ones((75000, 250), dtype=np.float64)
C = np.dot(S.T, S)
C /= np.linalg.norm(C, ord=2)
w, v = np.linalg.eig(C)
for ii in range(25):
    I = np.dot(S, v)

Information


Environment (conda list):

``` $ conda list # packages in environment at $HOME/miniconda3/envs/tissue_motion: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 1_llvm conda-forge asttokens 2.0.3 pypi_0 pypi attrs 19.3.0 py_0 conda-forge backcall 0.1.0 py_0 conda-forge bleach 3.1.1 py_0 conda-forge bzip2 1.0.8 h516909a_2 conda-forge ca-certificates 2019.11.28 hecc5488_0 conda-forge cairo 1.16.0 hfb77d84_1002 conda-forge certifi 2019.11.28 py38_0 conda-forge cffi 1.14.0 py38hd463f26_0 conda-forge chardet 3.0.4 py38_1003 conda-forge cloudpickle 1.3.0 py_0 conda-forge colorama 0.4.3 pypi_0 pypi conda 4.8.2 py38_0 conda-forge conda-package-handling 1.6.0 py38h516909a_1 conda-forge cryptography 2.8 py38h72c5cf5_1 conda-forge cycler 0.10.0 py_2 conda-forge cytoolz 0.10.1 py38h516909a_0 conda-forge dask-core 2.12.0 py_0 conda-forge dbus 1.13.6 he372182_0 conda-forge decorator 4.4.2 py_0 conda-forge defusedxml 0.6.0 py_0 conda-forge entrypoints 0.3 py38_1000 conda-forge executing 0.4.2 pypi_0 pypi expat 2.2.9 he1b5a44_2 conda-forge ffmpeg 4.1.3 h167e202_0 conda-forge fontconfig 2.13.1 h86ecdb6_1001 conda-forge freetype 2.10.0 he983fc9_1 conda-forge gettext 0.19.8.1 hc5be6a0_1002 conda-forge giflib 5.2.1 h516909a_2 conda-forge glib 2.58.3 py38h6f030ca_1002 conda-forge gmp 6.2.0 he1b5a44_2 conda-forge gnutls 3.6.5 hd3a4fd2_1002 conda-forge graphite2 1.3.13 hf484d3e_1000 conda-forge gst-plugins-base 1.14.5 h0935bb2_2 conda-forge gstreamer 1.14.5 h36ae1b5_2 conda-forge harfbuzz 2.4.0 h9f30f68_3 conda-forge hdf5 1.10.5 nompi_h3c11f04_1104 conda-forge icecream 2.0.0 pypi_0 pypi icu 64.2 he1b5a44_1 conda-forge idna 2.9 py_1 conda-forge imageio 2.8.0 py_0 conda-forge importlib_metadata 1.5.0 py38_0 conda-forge ipykernel 5.1.4 py38h5ca1d4c_0 conda-forge ipympl 0.5.3 py_0 conda-forge ipython 7.13.0 py38h5ca1d4c_0 conda-forge ipython_genutils 0.2.0 py_1 conda-forge ipywidgets 7.5.1 py_0 conda-forge jasper 1.900.1 h07fcdf6_1006 conda-forge jedi 0.16.0 py38_0 conda-forge jinja2 2.11.1 py_0 conda-forge joblib 0.14.1 py_0 conda-forge jpeg 9c h14c3975_1001 conda-forge jsonschema 3.2.0 py38_0 conda-forge jupyter_client 6.0.0 py_0 conda-forge jupyter_core 4.6.3 py38_0 conda-forge kiwisolver 1.1.0 py38hc9558a2_0 conda-forge lame 3.100 h14c3975_1001 conda-forge ld_impl_linux-64 2.33.1 h53a641e_8 conda-forge libblas 3.8.0 15_openblas conda-forge libcblas 3.8.0 15_openblas conda-forge libclang 9.0.1 default_hde54327_0 conda-forge libffi 3.2.1 he1b5a44_1006 conda-forge libgcc-ng 9.2.0 h24d8f2e_2 conda-forge libgfortran-ng 7.3.0 hdf63c60_5 conda-forge libiconv 1.15 h516909a_1005 conda-forge liblapack 3.8.0 15_openblas conda-forge liblapacke 3.8.0 15_openblas conda-forge libllvm9 9.0.1 hc9558a2_0 conda-forge libopenblas 0.3.8 h5ec1e0e_0 conda-forge libopencv 4.2.0 py38_2 conda-forge libpng 1.6.37 hed695b0_0 conda-forge libsodium 1.0.17 h516909a_0 conda-forge libstdcxx-ng 9.2.0 hdf63c60_2 conda-forge libtiff 4.1.0 hc3755c2_3 conda-forge libuuid 2.32.1 h14c3975_1000 conda-forge libwebp 1.0.2 h56121f0_5 conda-forge libxcb 1.13 h14c3975_1002 conda-forge libxkbcommon 0.10.0 he1b5a44_0 conda-forge libxml2 2.9.10 hee79883_0 conda-forge llvm-openmp 9.0.1 hc9558a2_2 conda-forge lz4-c 1.8.3 he1b5a44_1001 conda-forge markupsafe 1.1.1 py38h516909a_0 conda-forge matplotlib-base 3.2.0 py38h250f245_1 conda-forge mistune 0.8.4 py38h516909a_1000 conda-forge nbconvert 5.6.1 py38_0 conda-forge nbformat 5.0.4 py_0 conda-forge ncurses 6.1 hf484d3e_1002 conda-forge nettle 3.4.1 h1bed415_1002 conda-forge networkx 2.4 py_0 conda-forge notebook 6.0.3 py38_0 conda-forge nspr 4.25 he1b5a44_0 conda-forge nss 3.47 he751ad9_0 conda-forge numpy 1.18.1 py38h95a1406_0 conda-forge olefile 0.46 py_0 conda-forge opencv 4.2.0 py38_2 conda-forge openh264 1.8.0 hdbcaa40_1000 conda-forge openssl 1.1.1d h516909a_0 conda-forge pandoc 2.9.2 0 conda-forge pandocfilters 1.4.2 py_1 conda-forge parso 0.6.2 py_0 conda-forge pcre 8.44 he1b5a44_0 conda-forge pexpect 4.8.0 py38_0 conda-forge pickleshare 0.7.5 py38_1000 conda-forge pillow 7.0.0 py38hefe7db6_0 conda-forge pip 20.0.2 py_2 conda-forge pixman 0.38.0 h516909a_1003 conda-forge progress 1.5 py_1 conda-forge prometheus_client 0.7.1 py_0 conda-forge prompt-toolkit 3.0.4 py_0 conda-forge prompt_toolkit 3.0.4 0 conda-forge pthread-stubs 0.4 h14c3975_1001 conda-forge ptyprocess 0.6.0 py_1001 conda-forge py-opencv 4.2.0 py38h5ca1d4c_2 conda-forge pycosat 0.6.3 py38h516909a_1002 conda-forge pycparser 2.20 py_0 conda-forge pydicom 1.4.2 py_0 conda-forge pygments 2.6.1 py_0 conda-forge pyhht 0.1.0 pypi_0 pypi pyopenssl 19.1.0 py_1 conda-forge pyparsing 2.4.6 py_0 conda-forge pyqt 5.12.3 py38hcca6a23_1 conda-forge pyqt5-sip 4.19.18 pypi_0 pypi pyqtwebengine 5.12.1 pypi_0 pypi pyrsistent 0.15.7 py38h516909a_0 conda-forge pysocks 1.7.1 py38_0 conda-forge python 3.8.2 h9d8adfe_4_cpython conda-forge python-dateutil 2.8.1 py_0 conda-forge python_abi 3.8 1_cp38 conda-forge pywavelets 1.1.1 py38hc1659b7_0 conda-forge pyzmq 19.0.0 py38h1768529_0 conda-forge qt 5.12.5 hd8c4c69_1 conda-forge readline 8.0 hf8c457e_0 conda-forge requests 2.23.0 py38_0 conda-forge ruamel_yaml 0.15.80 py38h516909a_1000 conda-forge scikit-image 0.16.2 py38hb3f55d8_0 conda-forge scipy 1.4.1 py38h921218d_0 conda-forge send2trash 1.5.0 py_0 conda-forge setuptools 46.0.0 py38_0 conda-forge six 1.14.0 py38_0 conda-forge sqlite 3.30.1 hcee41ef_0 conda-forge terminado 0.8.3 py38_0 conda-forge testpath 0.4.4 py_0 conda-forge tk 8.6.10 hed695b0_0 conda-forge toolz 0.10.0 py_0 conda-forge tornado 6.0.4 py38h516909a_0 conda-forge tqdm 4.43.0 py_0 conda-forge traitlets 4.3.3 py38_0 conda-forge urllib3 1.25.7 py38_0 conda-forge wcwidth 0.1.8 py_0 conda-forge webencodings 0.5.1 py_1 conda-forge wheel 0.34.2 py_1 conda-forge widgetsnbextension 3.5.1 py38_0 conda-forge x264 1!152.20180806 h14c3975_0 conda-forge xorg-kbproto 1.0.7 h14c3975_1002 conda-forge xorg-libice 1.0.10 h516909a_0 conda-forge xorg-libsm 1.2.3 h84519dc_1000 conda-forge xorg-libx11 1.6.9 h516909a_0 conda-forge xorg-libxau 1.0.9 h14c3975_0 conda-forge xorg-libxdmcp 1.1.3 h516909a_0 conda-forge xorg-libxext 1.3.4 h516909a_0 conda-forge xorg-libxrender 0.9.10 h516909a_1002 conda-forge xorg-renderproto 0.11.1 h14c3975_1002 conda-forge xorg-xextproto 7.3.0 h14c3975_1002 conda-forge xorg-xproto 7.0.31 h14c3975_1007 conda-forge xz 5.2.4 h14c3975_1001 conda-forge yaml 0.2.2 h516909a_1 conda-forge zeromq 4.3.2 he1b5a44_2 conda-forge zipp 3.1.0 py_0 conda-forge zlib 1.2.11 h516909a_1006 conda-forge zstd 1.4.4 h3b9ef0a_1 conda-forge ```


Details about conda and system ( conda info ):

``` $ conda info active environment : npcrash active env location : $HOME/miniconda3/envs/npcrash shell level : 2 user config file : $HOME/.condarc populated config files : $HOME/.condarc conda version : 4.8.2 conda-build version : not installed python version : 3.7.6.final.0 virtual packages : __cuda=10.1 __glibc=2.30 base environment : $HOME/miniconda3 (writable) channel URLs : https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : $HOME/miniconda3/pkgs $HOME/.conda/pkgs envs directories : $HOME/miniconda3/envs $HOME/.conda/envs platform : linux-64 user-agent : conda/4.8.2 requests/2.23.0 CPython/3.7.6 Linux/5.3.0-40-generic ubuntu/19.10 glibc/2.30 UID:GID : 1000:1000 netrc file : None offline mode : False ```
isuruf commented 4 years ago

Please check 0.3.10