mkl-2017.0.4 / win-64 breaks numpy 1.12.1 and scipy 0.19.1

[wjp]

Certain parts of scipy fail with the new mkl-2017.0.4 package on 64 bit Windows.

(test) C:\Users\wjp>conda create -n testmkl1 python=3.6 numpy=1.12 mkl=2017.0.4 scipy
Fetching package metadata .............
Solving package specifications: .

Package plan for installation in environment D:\wjp\miniconda3\envs\testmkl1:

The following NEW packages will be INSTALLED:

    certifi:        2017.7.27.1-py36h043bc9e_0
    intel-openmp:   2018.0.0-hcd89f80_7
    mkl:            2017.0.4-h6d528fc_0
    numpy:          1.12.1-py36_0
    pip:            9.0.1-py36h226ae91_4
    python:         3.6.3-h210ce5f_2
    scipy:          0.19.1-np112py36_0
    setuptools:     36.5.0-py36h65f9e6e_0
    vc:             14-h2379b0c_2
    vs2015_runtime: 14.0.25123-hd4c4e62_2
    wheel:          0.29.0-py36h6ce6cde_1
    wincertstore:   0.2-py36h7fe50ca_0

Proceed ([y]/n)? y

#
# To activate this environment, use:
# > activate testmkl1
#
# To deactivate an active environment, use:
# > deactivate
#
# * for power-users using bash, you must source
#

(test) C:\Users\wjp>activate testmkl1

(testmkl1) C:\Users\wjp>python -c "import scipy.sparse.linalg"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\wjp\miniconda3\envs\testmkl1\lib\site-packages\scipy\sparse\linalg\__init__.py", line 113, in <module>
    from .isolve import *
  File "D:\wjp\miniconda3\envs\testmkl1\lib\site-packages\scipy\sparse\linalg\isolve\__init__.py", line 6, in <module>
    from .iterative import *
  File "D:\wjp\miniconda3\envs\testmkl1\lib\site-packages\scipy\sparse\linalg\isolve\iterative.py", line 7, in <module>
    from . import _iterative
ImportError: DLL load failed: The specified module could not be found.

(testmkl1) C:\Users\wjp>

Compare with mkl-2017.0.3:

(D:\wjp\miniconda3) C:\Users\wjp>conda create -n testmkl2 python=3.6 numpy=1.12
mkl=2017.0.3 scipy
Fetching package metadata .............
Solving package specifications: .

Package plan for installation in environment D:\wjp\miniconda3\envs\testmkl2:

The following NEW packages will be INSTALLED:

    certifi:        2017.7.27.1-py36h043bc9e_0
    mkl:            2017.0.3-0
    numpy:          1.12.1-py36_0
    pip:            9.0.1-py36h226ae91_4
    python:         3.6.3-h210ce5f_2
    scipy:          0.19.1-np112py36_0
    setuptools:     36.5.0-py36h65f9e6e_0
    vc:             14-h2379b0c_2
    vs2015_runtime: 14.0.25123-hd4c4e62_2
    wheel:          0.29.0-py36h6ce6cde_1
    wincertstore:   0.2-py36h7fe50ca_0

Proceed ([y]/n)? y

#
# To activate this environment, use:
# > activate testmkl2
#
# To deactivate an active environment, use:
# > deactivate
#
# * for power-users using bash, you must source
#

(D:\wjp\miniconda3) C:\Users\wjp>activate testmkl2

(testmkl2) C:\Users\wjp>python -c "import scipy.sparse.linalg"

(testmkl2) C:\Users\wjp>

Interestingly, if I upgrade to numpy-1.13 in the broken environment, and then downgrade to numpy-1.12 again (all in the same environment), then it does work. This numpy/mkl/scipy upgrade + downgrade seems to leave a number of dlls behind in the environment.

[msarahan]

What is your constraint here? You need to use numpy 1.12? There are newer scipy packages built against mkl 2018. Can you use those?

There is a problem here, but I'm not sure how important it is to fix.

[wjp]

Thanks for your reply. I am packaging a library for conda ( at https://anaconda.org/astra-toolbox/repo ), and I provide packages for a number of older python/numpy version combinations for compatibility. Some of our users still use numpy 1.12 due to 3rd party dependencies.

[msarahan]

OK. I'll try to figure this out soon. In the meantime, you should also consider building your packages so that they don't depend on a specific numpy version. Docs at https://conda-forge.org/docs/meta.html#building-against-numpy

[msarahan]

We have rebuilt numpy 1.12.1 against mkl 2018. Creating the env you show above now picks up all of the MKL 2018-built stuff, so it should be fine. I hope this is a viable solution for you. Please close this issue if it works.

[wjp]

Thank you for the quick fix! This works on my Windows system. Also thanks for the pointer on numpy; I'll read up on the forward compatibility provided.