Closed hustc12 closed 6 years ago
Could you please check which version of mkl is installed? is it 11.3.3 or 11.3.1?
Using miniconda on windows (no matter if 32 or 64 bits) one can not import linalg extensions anymore:
https://ci.appveyor.com/project/marscher/pyemma/build/1.0.1977/job/v6yddjbtdg5w6mq9#L1321
Thanks @marcher, it confirms my problem.
We see these scipy failures when mkl 11.3.3 is installed, but it worked fine with 11.3.1. I don't know whether it is a conda, mkl or a scipy issue. The version update happened sometime yesterday.
And this is an issue only affecting windows, but not linux or osx.
maybe this is also related to #753?
Maybe, I'm not a windows expert.
I'm also posting a traceback here, hopefully it will give clue for someone fixing this issue. Full log e.g. https://ci.appveyor.com/project/Astropy/astropy/build/1.0.4397/job/m7w5lahc0o37in8e
_______ [doctest] astropy.stats.info_theory.bayesian_info_criterion_lsq _______
160 >>> from astropy.stats.info_theory import bayesian_info_criterion_lsq
161 >>> # Generate fake data
162 >>> np.random.seed(0)
163 >>> x = np.linspace(-5., 5., 200)
164 >>> y = 3 * np.exp(-0.5 * (x - 1.3)**2 / 0.8**2)
165 >>> y += np.random.normal(0., 0.2, x.shape)
166 >>> # Fit the data using a Box model
167 >>> t_init = models.Trapezoid1D(amplitude=1., x_0=0., width=1., slope=0.5)
168 >>> fit_t = fitting.LevMarLSQFitter()
169 >>> t = fit_t(t_init, x, y)
UNEXPECTED EXCEPTION: ImportError('DLL load failed: The specified module could not be found.',)
Traceback (most recent call last):
File "C:\conda\envs\test\lib\doctest.py", line 1315, in __run
compileflags, 1) in test.globs
File "<doctest astropy.stats.info_theory.bayesian_info_criterion_lsq[9]>", line 1, in <module>
File "astropy\modeling\fitting.py", line 449, in __call__
from scipy import optimize
File "C:\conda\envs\test\lib\site-packages\scipy\optimize\__init__.py", line 232, in <module>
from .optimize import *
File "C:\conda\envs\test\lib\site-packages\scipy\optimize\optimize.py", line 37, in <module>
from .linesearch import (line_search_wolfe1, line_search_wolfe2,
File "C:\conda\envs\test\lib\site-packages\scipy\optimize\linesearch.py", line 18, in <module>
from scipy.optimize import minpack2
ImportError: DLL load failed: The specified module could not be found.
astropy\stats\info_theory.py:169: UnexpectedException
Seeing this on CI for Miniconda3, though not for Miniconda2.
[00:00:27] Installing
[00:00:27]
[00:00:27] C:\projects\pycalphad\Miniconda3-latest-Windows-x86_64.exe
[00:00:27]
[00:00:27] to
[00:00:27]
[00:00:27] C:\Python35-conda64
[00:00:27]
[00:00:27] C:\projects\pycalphad\Miniconda3-latest-Windows-x86_64.exe
[00:00:27]
[00:00:27] /S /D=C:\Python35-conda64
[00:00:27]
[00:00:27]
[00:00:27] Handles NPM(K) PM(K) WS(K) VM(M) CPU(s) Id ProcessName
[00:00:27] ------- ------ ----- ----- ----- ------ -- -----------
[00:00:27] 0 2 596 108 4 0.00 2696 Miniconda3-latest-...
[00:00:50] Python 3.5 (64) installation complete
[00:00:50]
[00:00:50] pip already installed.
[00:00:50]
[00:00:50]
[00:00:50] SET PATH=%PYTHON%;%PYTHON%\Scripts;%PATH%
[00:00:50] python --version
[00:00:50] Python 3.5.1 :: Continuum Analytics, Inc.
[00:00:50] python -c "import struct; print(struct.calcsize('P') * 8)"
[00:00:50] 64
[00:00:50] conda create --yes -n condaenv python=%PYTHON_VERSION%
[00:00:51] Fetching package metadata:
[00:00:54] Solving package specifications: .........
[00:01:13]
[00:01:13] Package plan for installation in environment C:\Python35-conda64\envs\condaenv:
[00:01:13]
[00:01:13] The following packages will be downloaded:
[00:01:13]
[00:01:13] package | build
[00:01:13] ---------------------------|-----------------
[00:01:13] setuptools-20.7.0 | py35_0 766 KB
[00:01:13]
[00:01:13] The following NEW packages will be INSTALLED:
[00:01:13]
[00:01:13] pip: 8.1.1-py35_1
[00:01:13] python: 3.5.1-4
[00:01:13] setuptools: 20.7.0-py35_0
[00:01:13] vs2015_runtime: 14.00.23026.0-0
[00:01:13] wheel: 0.29.0-py35_0
[00:01:13]
[snip]
[00:01:13] #
[00:01:13] # To activate this environment, use:
[00:01:13] # > activate condaenv
[00:01:13] #
[00:01:13] activate condaenv
[00:01:13] Deactivating environment "C:\Python35-conda64"...
[00:01:13] Activating environment "C:\Python35-conda64\envs\condaenv"...
[00:01:13] conda install --yes -n condaenv --quiet pip setuptools nose numpy pandas scipy sympy pyparsing matplotlib xarray
[00:01:14] Fetching package metadata:
[00:01:15] Solving package specifications: ..
[00:03:02] 1 file(s) copied.
[00:03:04] Package plan for installation in environment C:\Python35-conda64\envs\condaenv:
[00:03:04]
[00:03:04] The following packages will be downloaded:
[00:03:04]
[00:03:04] package | build
[00:03:04] ---------------------------|-----------------
[00:03:04] mkl-11.3.3 | 0 100.3 MB
[00:03:04] jpeg-8d | vc14_0 177 KB
[00:03:04] libpng-1.6.17 | vc14_1 464 KB
[00:03:04] openssl-1.0.2h | vc14_0 5.1 MB
[00:03:04] tk-8.5.18 | vc14_0 2.2 MB
[00:03:04] zlib-1.2.8 | vc14_2 109 KB
[00:03:04] libtiff-4.0.6 | vc14_1 461 KB
[00:03:04] mpmath-0.19 | py35_0 402 KB
[00:03:04] nose-1.3.7 | py35_0 242 KB
[00:03:04] numpy-1.11.0 | py35_1 2.8 MB
[00:03:04] pyparsing-2.1.1 | py35_0 69 KB
[00:03:04] pytz-2016.4 | py35_0 171 KB
[00:03:04]
[00:03:04] sip-4.16.9 | py35_2 229 KB
[00:03:04] six-1.10.0 | py35_0 8 KB
[00:03:04] cycler-0.10.0 | py35_0 11 KB
[00:03:04] python-dateutil-2.5.3 | py35_0 237 KB
[00:03:04] qt-4.8.7 | vc14_7 52.1 MB
[00:03:04] scipy-0.17.1 | np111py35_0 10.6 MB
[00:03:04] sympy-1.0 | py35_0 6.4 MB
[00:03:04] pandas-0.18.1 | np111py35_0 6.9 MB
[00:03:04] pyqt-4.11.4 | py35_5 3.8 MB
[00:03:04] matplotlib-1.5.1 | np111py35_0 6.2 MB
[00:03:04] xarray-0.7.2 | py35_0 334 KB
[00:03:04] ------------------------------------------------------------
[00:03:04] Total: 199.3 MB
[00:03:04]
[00:03:04] The following NEW packages will be INSTALLED:
[00:03:04]
[00:03:04] cycler: 0.10.0-py35_0
[00:03:04] jpeg: 8d-vc14_0 [vc14]
[00:03:04] libpng: 1.6.17-vc14_1 [vc14]
[00:03:04] libtiff: 4.0.6-vc14_1 [vc14]
[00:03:04] matplotlib: 1.5.1-np111py35_0
[00:03:04] mkl: 11.3.3-0
[00:03:04] mpmath: 0.19-py35_0
[00:03:04] nose: 1.3.7-py35_0
[00:03:04] numpy: 1.11.0-py35_1
[00:03:04] openssl: 1.0.2h-vc14_0 [vc14]
[00:03:04] pandas: 0.18.1-np111py35_0
[00:03:04] pyparsing: 2.1.1-py35_0
[00:03:04] pyqt: 4.11.4-py35_5
[00:03:04] python-dateutil: 2.5.3-py35_0
[00:03:04] pytz: 2016.4-py35_0
[00:03:04] qt: 4.8.7-vc14_7 [vc14]
[00:03:04] scipy: 0.17.1-np111py35_0
[00:03:04] sip: 4.16.9-py35_2
[00:03:04] six: 1.10.0-py35_0
[00:03:04] sympy: 1.0-py35_0
[00:03:04] tk: 8.5.18-vc14_0 [vc14]
[00:03:04] xarray: 0.7.2-py35_0
[00:03:04] zlib: 1.2.8-vc14_2 [vc14]
[00:03:04]
[00:03:05] conda install --yes -n condaenv --quiet -c richardotis tinydb autograd tqdm
[00:03:05] Fetching package metadata:
[00:03:08] Solving package specifications:
[00:03:12] Package plan for installation in environment C:\Python35-conda64\envs\condaenv:
[00:03:12]
[00:03:12] The following packages will be downloaded:
[00:03:12]
[00:03:12] package | build
[00:03:12] ---------------------------|-----------------
[00:03:12] future-0.15.2 | py35_0 307 KB
[00:03:12] tinydb-2.4 | py35_0 63 KB
[00:03:12] tqdm-3.8.0 | py35_0 37 KB
[00:03:12] autograd-1.1.1 | py35_0 74 KB
[00:03:12] ------------------------------------------------------------
[00:03:12] Total: 480 KB
[00:03:12]
[00:03:12] The following NEW packages will be INSTALLED:
[00:03:12]
[00:03:12] autograd: 1.1.1-py35_0
[00:03:12] future: 0.15.2-py35_0
[00:03:12] tinydb: 2.4-py35_0
[00:03:12] tqdm: 3.8.0-py35_0
[00:03:12]
[snip]
[00:03:15] Successfully installed pycalphad
[00:03:15]
[00:03:16] You are using pip version 8.1.1, however version 8.1.2 is available.
[00:03:16] You should consider upgrading via the 'python -m pip install --upgrade pip' command.
[00:03:16]
[00:03:16] python -c "import nose ; nose.main()" -s -v pycalphad
[00:03:20] Failure: ImportError (DLL load failed: The specified module could not be found.) ...
[00:03:20] ERROR
[00:03:20]
[00:03:20] ======================================================================
[00:03:20] ERROR: Failure: ImportError (DLL load failed: The specified module could not be found.)
[00:03:20] ----------------------------------------------------------------------
[00:03:20] Traceback (most recent call last):
[00:03:20] File "C:\Python35-conda64\envs\condaenv\lib\site-packages\nose\failure.py", line 39, in runTest
[00:03:20] raise self.exc_val.with_traceback(self.tb)
[00:03:20] File "C:\Python35-conda64\envs\condaenv\lib\site-packages\nose\loader.py", line 418, in loadTestsFromName
[00:03:20] addr.filename, addr.module)
[00:03:20] File "C:\Python35-conda64\envs\condaenv\lib\site-packages\nose\importer.py", line 47, in importFromPath
[00:03:20] return self.importFromDir(dir_path, fqname)
[00:03:20] File "C:\Python35-conda64\envs\condaenv\lib\site-packages\nose\importer.py", line 94, in importFromDir
[00:03:20] mod = load_module(part_fqname, fh, filename, desc)
[00:03:20] File "C:\Python35-conda64\envs\condaenv\lib\imp.py", line 244, in load_module
[00:03:20] return load_package(name, filename)
[00:03:20] File "C:\Python
[00:03:20] 35-conda64\envs\condaenv\lib\imp.py", line 216, in load_package
[00:03:20] return _load(spec)
[00:03:20] File "<frozen importlib._bootstrap>", line 693, in _load
[00:03:20] File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
[00:03:20] File "<frozen importlib._bootstrap_external>", line 662, in exec_module
[00:03:20] File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
[00:03:20] File "C:\projects\pycalphad\pycalphad\__init__.py", line 9, in <module>
[00:03:20] from pycalphad.core.equilibrium import equilibrium
[00:03:20] File "C:\projects\pycalphad\pycalphad\core\equilibrium.py", line 19, in <module>
[00:03:20] import scipy.spatial
[00:03:20] File "C:\Python35-conda64\envs\condaenv\lib\site-packages\scipy\spatial\__init__.py", line 95, in <module>
[00:03:20] from ._procrustes import procrustes
[00:03:20] File "C:\Python35-conda64\envs\condaenv\lib\site-packages\scipy\spatial\_procrustes.py", line 11, in <module>
[00:03:20] from scipy.linalg import orthogonal_procrustes
[00:03:20] File "C:\Python35-conda64\envs\condaenv\lib\site-packages\scipy\linalg\__init__.py", line 19
[00:03:20] 0, in <module>
[00:03:20] from ._decomp_update import *
[00:03:20] File "scipy/linalg/_decomp_update.pyx", line 1, in init scipy.linalg._decomp_update (scipy\linalg\_decomp_update.c:39400)
[00:03:20] ImportError: DLL load failed: The specified module could not be found.
[00:03:20]
[00:03:20] ----------------------------------------------------------------------
[00:03:20] Ran 1 test in 0.000s
[00:03:20]
[00:03:20] FAILED (errors=1)
[00:03:20] Command exited with code 1
@bsipocz , seems strange:
import mkl mkl.version '1.1.2'
BTW, I am not using Miniconda, I am using a Windows 64-bit Graphical Installer (http://repo.continuum.io/archive/Anaconda2-4.0.0-Windows-x86_64.exe), and all my 3 windows platform encountered this issue under this installer.
@kevin-coder import mkl
will import mkl-service - if installed. This is not the version of your mkl
library.
@MSeifert04 , thank you for pointing out my wrong doing. When I am trying to update the mkl, following information printed out:
The following packages will be UPDATED:
mkl: 11.3.1-0 --> 11.3.3-0
mkl-service: 1.1.2-py27_0 --> 1.1.2-py27_1
numexpr: 2.5-np110py27_0 --> 2.5.2-np110py27_1
numpy: 1.10.4-py27_0 --> 1.10.4-py27_2
scikit-learn: 0.17.1-np110py27_0 --> 0.17.1-np110py27_1
scipy: 0.17.0-np110py27_0 --> 0.17.0-np110py27_4
So I think currently the mkl on my machine is 11.3.1, I will update it to the latest version to have a look first. If the same issue still exists, I will report.
I can reproduce this by just installing miniconda, running conda install scipy
and then running python -c "import scipy.linalg"
. I see it in the latest Python 3.5 builds (both built as of 5/13/16) for scipy 0.17.1 and 0.17.0. I haven't tested with Python 2.7 yet. Looking at the compiled modules with dependency walker doesn't show any missing dependencies though.
I'm not set up with ifort and the old build system to build this package myself, so further debugging will probably need direct attention from @ilanschnell.
https://github.com/ContinuumIO/anaconda-issues/issues/793 appears to be a duplicate.
Given that all the new builds use the latest MKL, one (temporary) workaround that fixes it for me is to do conda install mkl=11.3.1
. That's not at all ideal though.
hi, all, After I update the scipy and mkl with conda update, there is no crash when I ran tests again, but there are ERRORs caused by DLL import fail, attached running log, please help to have a look. Thanks! Logs: scipyTests.txt
Thanks for the feedback. Indeed, some DLLs appear to be missing in the new MKL conda packages on Windows. I will take a closer look.
Thanks again Ilan
On Sun, May 15, 2016 at 10:31 PM, Zhenqing, Hu notifications@github.com wrote:
hi, all, After I update the scipy and mkl with conda update, there is no crash when I ran tests again, but there are fails caused by DLL import fail, attached running log, please help to have a look. Thanks! scipyTests.txt https://github.com/ContinuumIO/anaconda-issues/files/265501/scipyTests.txt
— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/ContinuumIO/anaconda-issues/issues/792#issuecomment-219341649
The problem should be fixed now. I've added the missing compiler related DLLs to the new MKL conda package.
@ilanschnell , thank you for quick response! But how could I get these missing DLLs? Seems that if I use "conda update xxx", it cannot work.
Oh, simply:
conda update mkl
Unfortunately, after updating mkl to version "11.3.3 1" and run test, crash again on my machine... Attach the test log as following, this crash looks similar with the one appeared at the very beginning on my side. scipyTest.txt
mkl 11.3.3-1 is all green for me on CI, thanks for the work.
Thanks for the fix!
OK, seems that I need a clean installation to have a try again... Update mkl on my side still cannot work. BTW, will we re-generate the Anaconda installer which include the latest mkl update again? If yes, I could have a clean installation and try again. Thanks!
Test Env: Windows 8.1 or Windows 10 Python 2.7.11 |Anaconda 4.0.0 (64-bit)
Reproduce Step:
Expected Results: At least the tests running should not crash
Actual Results: Tests will crash.
Investigation details: https://github.com/scipy/scipy/issues/6158