search

spyder-ide / spyder

Official repository for Spyder - The Scientific Python Development Environment
https://www.spyder-ide.org
MIT License
8.19k stars 1.58k forks source link

Clean conda Windows install, followed by 'conda update --all' results in wrong mkl dlls being used for Spyder #7357

Closed luciansmith closed 5 years ago

luciansmith commented 6 years ago

Problem Description

A clean conda install from https://www.anaconda.com/download/#windows (version 5.2.0) works for Spyder, but if I then do a 'conda update --all', the resulting configuration is broken. When you run Spyder, launching from Anaconda navigator, you get a message about a missing ordinal 242 in Anaconda3\Library\bin\mkl_intel_thread.dll

This turns out to be because the dlls from mkl-2018.0.3-1 are installed in Anaconda3\Library\bin\, but something in Spyder needs the dlls from mkl-2018.0.2-1 instead. If you copy the dlls from Anaconda3\pkgs\mkl-2018.0.2-1\Library\bin into Anaconda3\Library\bin, Spyder then launches normally. (This probably breaks something else, but if so, it's something I haven't yet tried to use ;-)

What steps reproduce the problem?

  1. Install Anaconda for Windows
  2. run conda update --all
  3. Launch Spyder

What is the expected output? What do you see instead?

Paste Traceback/Error Below (if applicable)


PASTE TRACEBACK HERE

Versions

Dependencies

pyflakes >=0.6.0 :  2.0.0 (OK)
pycodestyle >=2.3:  2.4.0 (OK)
pygments >=2.0   :  2.2.0 (OK)
pandas >=0.13.1  :  0.23.1 (OK)
numpy >=1.7      :  1.14.5 (OK)
sphinx >=0.6.6   :  1.7.5 (OK)
rope >=0.9.4     :  0.10.7 (OK)
jedi >=0.9.0     :  0.12.0 (OK)
nbconvert >=4.0  :  5.3.1 (OK)
sympy >=0.7.3    :  1.1.1 (OK)
cython >=0.21    :  0.28.3 (OK)
qtconsole >=4.2.0:  4.3.1 (OK)
IPython >=4.0    :  6.4.0 (OK)
pylint >=0.25    :  1.9.2 (OK)
msarahan commented 5 years ago

@mingwandroid I believe it got through testing because as @luciansmith described, the installer itself is fine. It is a self-consistent set. Somehow updating brings in some other software that might be built against mkl 2018, and that triggers the issue. I'm not sure exactly why an update would use an older MKL, but I'll try to reproduce.

CAM-Gerlach commented 5 years ago

Thanks so much @msarahan ! I really appreciate it! Let me know if you need more testing data from me.

JimVee commented 5 years ago

@janm97 Did you try conda install mkl=2018.0.2?

This fixed my problem. Thank-you.

rpcoelho17 commented 5 years ago

I'm having the same problem. Just did a brand new installation of Anaconda 3.7, then tried installing spyder from Anaconda Navigator and got the error mentioned above. I tried using the "Conda install mkl=2018.0.2" mentioned above and got the following error:

(base) C:\Users\Rodrigo>conda install mkl=2018.0.2 Solving environment: failed

UnsatisfiableError: The following specifications were found to be in conflict:

mingwandroid commented 5 years ago

Spyder comes pre-installed with Anaconda and therefore does not need to be installed. Did you use Minicoda instead here?

rpcoelho17 commented 5 years ago

No I used the full Anaconda. I appologize, the error occurs when you click launch in the Anaconda Navigator to run spyder. image

CAM-Gerlach commented 5 years ago

got the error mentioned above.

As you can see from your error message, this is not the exact same error, as the mentioned DLL is different. Post the full output of where libiomp5md.dll run in Anaconda prompt here; it is possible that it is due to another program's DLLs conflicting with the indicated one rather than anything to do with mkl itself.

UnsatisfiableError: The following specifications were found to be in conflict:

This is due to mkl < 2018.0.3 not being built for Python 3.7. You can try the above suggestions to fix the root problem, and if that doesn't work do conda install python=3.6 mkl=2018.0.2 though you'll be stuck on Python 3.6 until the root problem (either with mkl or with your system, whichever is the case here) is fixed.

you click launch in the Anaconda Navigator to run spyder.

FYI, AFAIK there is no advantage to launch Spyder via the method, as it is vastly faster to merely launch it via the Start menu shortcut on Windows like with any other application (just tap Win and start typing spyder ) than going through the extra layer of complexity that Navigator poses as well as potential for additional errors and problems, unless you need to launch it in a specific environment and don't want to use Anaconda prompt for that.

rpcoelho17 commented 5 years ago

Here is the output of the were command in the Anaconda prompt:

(base) C:\Users\Rodrigo>where libiomp5md.dll C:\Users\Rodrigo\Anaconda3\Library\bin\libiomp5md.dll C:\Windows\System32\libiomp5md.dll

Running spider directly from windows (outside of Navigator) generates the same error.

Will running "Conda install python=3.6 mkl=2018.0.2" replace the current 3.7 installation or create a new environment? I would like to avoid having many environments and would be ok with having just 3.6 on my machine.

Thanks for the help, by the way! :)

CAM-Gerlach commented 5 years ago

Here is the output of the were command in the Anaconda prompt:

Thanks. As you can see, another program on your machine has installed a DLL with an identical name in directly in the System32 directory, which is a very bad practice and a recipe for all sorts of problems. I'm not sure what program it could be, but there are various ways to find out. In any case, there is nothing we can do about this on our end; this needs to be fixed on your machine by uninstalling the offending other program, removing the DLL (and seeing what breaks), etc.

Will running "Conda install python=3.6 mkl=2018.0.2" replace the current 3.7 installation or create a new environment?

It will modify the current default (base) environment.

mingwandroid commented 5 years ago

I'm not sure what program it could be, but there are various ways to find out.

I would bet that the program in question is called Amplitube. I've tried to contact the developers of this software to ask them not to do this egregious act but have been unable to get any response from them. I might have to renew my efforts. Ideally MKL's redistribution license would prohibit this!

mingwandroid commented 5 years ago

https://mobile.twitter.com/_tmp_account/status/1083817363047370754

Please help to spread this if you can!

CAM-Gerlach commented 5 years ago

I would bet that the program in question is called Amplitube.

I thought the problematic DLLs with Amplitube were mkl_intel_core, mkl_intel_thread and mkl_def so I wasn't sure if there were others, but I guess so if it uses all of mkl.

Ideally MKL's redistribution license would prohibit this!

Have you considered mentioning it to your Intel contacts from that end?

msarahan commented 5 years ago

We have. We're trying to figure out DLL load tricks that will prevent the problem. We will also explore application manifests, but that's hard to get right - you'd have to re-generate the whole manifest for all libraries in an env any time that any of them were updated.

CAM-Gerlach commented 5 years ago

Okay, great—thanks for the update @msarahan . Meanwhile, any progress figuring out what's going on in the cases like mine where there aren't any conflicting DLLs in System32 (or anywhere else on the where path) that could be causing the problem? Anything else you want me to try? Thanks!

bverem commented 5 years ago

I have a fresh Anaconda (2018.12) install on Windows 10 with this issue, and I do not have Amplitube installed (never heard of it). The fix for me was finding libiomp5md.dll in my system32 folder and renaming it. No idea which program uses it, but hopefully I don't have to find out. Otherwise, simply reverting to mkl 2018.0.2 and python 3.6 didn't completely help (I could at least import numpy, but certain functions still caused errors on loading).

chucku commented 5 years ago

I have a fresh Anaconda (2018.12) install on Windows 10 with this issue, and I do not have Amplitube installed (never heard of it). The fix for me was finding libiomp5md.dll in my system32 folder and renaming it. No idea which program uses it, but hopefully I don't have to find out. Otherwise, simply reverting to mkl 2018.0.2 and python 3.6 didn't completely help (I could at least import numpy, but certain functions still caused errors on loading).

I had the same problem "missing ordinal 242 in Anaconda3\Library\bin\mkl_intel_thread.dll" and renaming the libiomp5md.dll file in the system32 folder has fixed it!

msarahan commented 5 years ago

@CAM-Gerlach sorry, I've got nothing. The one other DLL we've seen causing this issue is the openmp runtime, libiomp5md.dll

CAM-Gerlach commented 5 years ago

Okay, thanks @msarahan . I guess I should just do a full, clean reinstall of Anaconda; I've just been putting it off since I don't want to have to re-create all my environments and such.

msarahan commented 5 years ago

I really don't think that will help. I think the conflict is with other software on your system. Reinstalling Anaconda won't fix those conflicts.

Ondrej-Martinek commented 5 years ago

Holy s%%t! Amplitube uninstalled and Python libraries suddenly work!!!!!! Thanks!!!!! Troubled me entire month.

"message about a missing ordinal 242 in Anaconda3\Library\bin\mkl_intel_thread.dll"

ChaoticRoman commented 5 years ago

Confirmed with the latest Anaconda Python 3 64-bit from www.anaconda.com/download/:

I have a way how to replicate this problem (with Anaconda Python in my PATH), just open cmd and

C:\Users\sobriety>python
Python 3.7.1 (default, Dec 10 2018, 22:54:23) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> import matplotlib.pyplot as plt
>>> plt.show()
>>> plt.plot([])
[<matplotlib.lines.Line2D object at 0x000001BD881038D0>]
>>> plt.show()

And we got "missing ordinal 242 in Anaconda3\Library\bin\mkl_intel_thread.dll".

Doesn't look like libiomp5md.dll problem:

C:\Users\sobriety>where libiomp5md.dll
C:\Users\sobriety\AppData\Local\Continuum\anaconda3\Library\bin\libiomp5md.dll

No amplitube present:

C:\Users\sobriety>conda remove amplitube
Solving environment: failed

PackagesNotFoundError: The following packages are missing from the target environment:
  - amplitube

This has fixed the problem:

Conda install python=3.6 mkl=2018.0.2
CAM-Gerlach commented 5 years ago

@ChaoticRoman Amplitube isn't an Anaconda package, so it naturally wouldn't be uninstalled by conda remove amplitube. It needs to be checked for removed through Programs and Features.

Doesn't look like libiomp5md.dll problem:

No, because the error you got was

missing ordinal 242 in Anaconda3\Library\bin\mkl_intel_thread.dll

Therefore, you should check where mkl_core.dll and where mkl_intel_thread.dll and post the results.

Also, check your Programs and Features list for Amplitube, or any of the other applications reported here. Thanks.

ChaoticRoman commented 5 years ago

Thank you for quick reply, CAM-Gerlach!

Therefore, you should check where mkl_core.dll and where mkl_intel_thread.dll and post the results.

Right now, after the Conda install python=3.6 mkl=2018.0.2, there is 

C:\Users\sobriety>where mkl_core.dll
C:\Users\sobriety\AppData\Local\Continuum\anaconda3\Library\bin\mkl_core.dll

C:\Users\sobriety>where mkl_intel_thread.dll
C:\Users\sobriety\AppData\Local\Continuum\anaconda3\Library\bin\mkl_intel_thread.dll

Should I somehow revert and try it again with failing bits in place?

Also, check your Programs and Features list for Amplitube, or any of the other applications reported here. Thanks.

No discussed programs seen, it is clean Windows 10 Pro x64 installation and only this has been installed:

  1. Install the latest Anaconda Python 3 64-bit
  2. Install the latest DAQNavi Driver for USB-4716
  3. Install the latest Advantech Multimeter
  4. Install Virtual COM Port emulator and create two virtual COM port pairs
  5. Install Mercury RT Dream Edition x64 using provided installator.
  6. Install IDS camera driver.
CAM-Gerlach commented 5 years ago

Thanks for the detailed reply, @ChaoticRoman . That's rather puzzling, indeed. Can you open the Anaconda prompt from the Start Menu, run the above commands in Python, and make sure you still see the error?

Assuming you do, can you post the output of echo %PATH% from Anaconda prompt in the environment in which you are running the above commands and getting the error (presumably base by default)?

The basic idea is you'll then want to search any of the directories identified on the %PATH% that aren't Anaconda-related for any .dlls that contain the string omp or mkl and report the results. I think I've actually found the culprit in my install, even though those commands came back clean. Thanks!

CAM-Gerlach commented 5 years ago

@mingwandroid @msarahan I finally isolated and resolved the issue on my machine; it was the report involving OpenMP that clued me in.

To make a long story short, I found a number of instances of OpenMP-related DLLs a few different places on my %PATH%, and after a good deal of testing, one in particular turned out to be the culprit: C:\Windows\System32\libiomp5md.dll, despite it not being referenced in the error message. Everything works fine now.

So, I can't be certain since I've uninstalled it many years ago, but some sleuthing revealed that GoPro Studio (at least older versions) appears to have been the probable original culprit here that people should check for. Also, interestingly enough, Intel Visual Fortran Redistributables on IA-32 (included with a proprietary scientific application, not sure which) also includes its DLLs in a common location which it places on the %PATH%; although it doesn't seem to trigger this particular problem, it has dozens of DLLs that are all in common with those in Library/bin/, as Anaconda no doubt uses the same Intel shared libraries for MKL/OpenMP/etc.

Also, FYI @ChaoticRoman you should be able to trigger the error by simply executing

import matplotlib.pyplot as plt
plt.plot([])

The other lines were not necessary, at least for me.

Here is "the long story" detailing the specifics of what I found, for anyone (morbidly) curious...

For the successfully working environment with mkl 2018.0.2 and Python 3.6.7 on Windows 8.1 x64 and the following abridged list of possibly relevant package versions:
``` blas 1.0 mkl icc_rt 2017.0.4 h97af966_0 intel-openmp 2018.0.0 8 matplotlib 3.0.2 py36hc8f65d3_0 mkl 2018.0.2 1 mkl-service 1.1.2 py36hb217b18_4 mkl_fft 1.0.1 py36h452e1ab_0 mkl_random 1.0.1 py36h9258bd6_0 mpmath 1.0.0 py36_2 numpy 1.14.3 py36h9fa60d3_1 numpy-base 1.14.3 py36h555522e_1 python 3.6.7 h33f27b4_1 vc 14.1 h0510ff6_3 vs2013_runtime 12.0.21005 1 vs2015_runtime 15.5.2 3 ```
...Running ``where libiomp5md.dll'' returns (version numbers of each DLL in brackets): ``` [5.0.2017.525] C:\Anaconda3\envs\spyder-4-lsp\Library\bin\libiomp5md.dll [5.0.2017.525] C:\Anaconda3\Library\bin\libiomp5md.dll [5.0.2014.710] C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\ia32\compiler\libiomp5md.dll # Evidently installed by Intel Visual Fortran Redistributables on IA-32, which was in turn installed by a proprietary scientific-related package (unsure which) [5.0.2009.811] C:\Windows\System32\libiomp5md.dll # Most likely related to the long-removed GoPro Studio video software, which I installed to get access to the CineForm professional codec, given I installed it around this time and it has a known issue with putting this DLL in System32 ``` ``where vcomp140.dll`` (VS2015) returns ``` [14.14.26405.0] C:\Anaconda3\envs\spyder-4-lsp\vcomp140.dll [14.14.26405.0] C:\Anaconda3\envs\spyder-4-lsp\Library\bin\vcomp140.dll [14.14.26405.0] C:\Anaconda3\vcomp140.dll [14.14.26405.0] C:\Anaconda3\Library\bin\vcomp140.dll [14.16.27012.6] C:\Windows\System32\vcomp140.dll # Installed by Visual Studio build tools which I installed as instructed to build Python C extensions; well after this problem first appeared ``` and ``where vcomp120.dll`` (VS2013) returns ``` [12.0.21005.1] C:\Anaconda3\envs\spyder-4-lsp\vcomp120.dll [12.0.21005.1] C:\Anaconda3\envs\spyder-4-lsp\Library\bin\vcomp120.dll [12.0.21005.1] C:\Program Files\ImageMagick-7.0.8-Q16\vcomp120.dll # Installed with ImageMagick and added to PATH more recently than problem first appeared [12.0.21005.1] C:\Windows\System32\vcomp120.dll # Possibly related to GoPro Studio or other such video encoding software installed around this time and long since removed ``` Searching all processes with Process Explorer for any using DLLs that include the string ``omp`` returned only Python processions using the Anaconda versions, so there doesn't appear to be anything too important using the other DLLs on the path at present. From here, if I run ``conda update mkl``, which resolves to ``mkl`` 2019.1 (144), Conda wants to *downgrade* ``numpy`` to 1.14.2 (from 1.14.3) while *upgrading* ``numpy-base`` to ``1.15.4`` (also from 1.14.3) (I didn't actually execute the change);
``` environment location: C:\Anaconda3\envs\spyder-4-lsp added / updated specs: - mkl The following packages will be downloaded: package | build ---------------------------|----------------- numpy-1.14.2 | py36h5c71026_0 3.7 MB numpy-base-1.15.4 | py36hc3f5095_0 3.9 MB mkl-2019.1 | 144 158.3 MB scipy-1.1.0 | py36h29ff71c_2 13.3 MB mkl-service-1.1.2 | py36hb782905_5 134 KB ------------------------------------------------------------ Total: 179.4 MB The following packages will be UPDATED: icc_rt: 2017.0.4-h97af966_0 --> 2019.0.0-h0cc432a_1 mkl: 2018.0.2-1 --> 2019.1-144 mkl-service: 1.1.2-py36hb217b18_4 --> 1.1.2-py36hb782905_5 numpy-base: 1.14.3-py36h555522e_1 --> 1.15.4-py36hc3f5095_0 scipy: 1.1.0-py36h672f292_0 --> 1.1.0-py36h29ff71c_2 The following packages will be DOWNGRADED: numpy: 1.14.3-py36h9fa60d3_1 --> 1.14.2-py36h5c71026_0 ```
``conda update numpy numpy-base`` states ``# All requested packages already installed.`` and ``conda install numpy=1.15.4`` spends about 5-10 minutes ``Solving environment`` and finally spits out ``` UnsatisfiableError: The following specifications were found to be in conflict: - mkl_fft=1.0.1 - numpy=1.15 -> mkl_fft[version='>=1.0.4'] ``` So then I executed ``conda install mkl_fft=1.0.4``, which also installed ``mkl`` 2018.0.3 and nothing else:
``` environment location: C:\Anaconda3\envs\spyder-4-lsp added / updated specs: - mkl_fft=1.0.4 The following packages will be downloaded: package | build ---------------------------|----------------- mkl_fft-1.0.4 | py36h1e22a9b_1 120 KB mkl-2018.0.3 | 1 178.1 MB ------------------------------------------------------------ Total: 178.2 MB The following packages will be UPDATED: mkl: 2018.0.2-1 --> 2018.0.3-1 mkl_fft: 1.0.1-py36h452e1ab_0 --> 1.0.4-py36h1e22a9b_1 ```
I can now reproduce the error (Ordinal 241/``mkl_intel_thread.dll``) in the Python interpreter by simply running ```python import matplotlib.pyplot as plt plt.plot([]) ``` I checked and none of the aforementioned DLL versions or files were modified, just the MKL ones. After testing changing the names of the various non-Anaconda DLLs, I found the culprit: ``C:\Windows\System32\libiomp5md.dll``. I can't be 100% sure since I've uninstalled the application itself many years ago, but I strongly suspect it was installed by GoPro Studio, a *gratis* video editing/encoding application which I once needed in order to get access to their Cineform professional video intermediate codec. The created/modified and last accessed dates roughly match up with when I installed and used it, and it has a known issue with installing this specific DLL in the System32 directory and causing conflicts with other video-related applications.
msarahan commented 5 years ago

Yep, other applications on PATH won't interfere with Anaconda Prompt, because Anaconda Prompt puts our entries up front. Doing that would be too disruptive as a global modification to PATH - that's why we recommend Anaconda Prompt. Running our other start menu shortcuts does similar stuff.

System32 is higher in priority than anything on PATH (https://docs.microsoft.com/en-us/windows/desktop/dlls/dynamic-link-library-search-order#standard-search-order-for-desktop-applications). It doesn't matter what we try to do with PATH if a 3rd party has put DLLs into System32. Manifests may still help us if we can figure that out someday.

Thansk for your patience, @CAM-Gerlach

ChaoticRoman commented 5 years ago

[...]open the Anaconda prompt from the Start Menu, run the above commands in Python, and make sure you still see the error?

Everything is fine after Conda install python=3.6 mkl=2018.0.2.

Assuming you do, can you post the output of echo %PATH% from Anaconda prompt in the environment in which you are running the above commands and getting the error (presumably base by default)?

OK, uninstalling Anaconda and installing it again. I have find out that we have these crashes in my application scripts but not while trying normal replication . But I have find out that if working directory is a place, where we have these scripts, it does crash:

C:\Program Files\MercuryRT-v2.7DE>python
Python 3.7.1 (default, Dec 10 2018, 22:54:23) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib
Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll.

C:\Program Files\MercuryRT-v2.7DE>

[...]any .dlls that contain the string omp or mkl and report the results. I think I've actually found the culprit in my install, even though those commands came back clean. Thanks!

C:\Program Files\MercuryRT-v2.7DE>dir *omp*.dll *mkl*.dll
 Volume in drive C has no label.
 Volume Serial Number is A432-6ABC

 Directory of C:\Program Files\MercuryRT-v2.7DE

08.10.2018  12:55           885 096 libiomp5md.dll
08.10.2018  12:56           176 512 vcomp140.dll
20.07.2018  11:01           152 064 vtkCommonComputationalGeometry-8.1.dll

So this seems to be the actual problem, not Anaconda itself, but running our scripts from bad location.

CAM-Gerlach commented 5 years ago

Everything is fine after conda install python=3.6 mkl=2018.0.2.

Sorry, what I said was very confusing. What I meant was to switch back to the current version of mkl and run the commands

import matplotlib.pyplot as plt
plt.plot([])

from the Anaconda prompt, not the regular command line, just in case there was some issue with your %PATH% that the latter would avoid. However, that's a moot point now.

can you post the output of echo %PATH% from Anaconda prompt

OK, uninstalling Anaconda and installing it again.

Sorry, what? How did "post the output of echo %PATH%" get interpreted as "reinstall Anaconda? As @msarahan said about my case, this isn't going to fix the problem if it is a conflict, as seemed likely was the case (and in fact was).

So this seems to be the actual problem, not Anaconda itself, but running our scripts from bad location.

Yes. The problem is that the DLL search path order will always search the current directory before the PATH, so it will find those DLLs instead of the correct Anaconda ones. To resolve this, just make sure not to run your scripts from the same directory as those conflicting DLLs, or you're likely to run into more issues in the future.

msarahan commented 5 years ago

Yep, and the ultimate solution is finding ways to overcome the problems with the search order, such as manifest files that specify particular versions, so that the globally installed ones won't get used if they are not compatible.

CAM-Gerlach commented 5 years ago

Yeah, I was wondering if there was a way to simply check the version string of the first found DLL and fall through to the next one if it doesn't match what the package is compiled against. But I figured it was Microsoft so it couldn't be that easy, heh.

mingwandroid commented 5 years ago

@CAM-Gerlach, we (Anaconda) have a potential workaround for the Windows DLL search order. It's in our latest python builds on the c3i_test2 anaconda.org channel:

python 2.7.15 build 7 3.6.8 build 1 3.7.2 build 1

Try updating python using the c3i_test2 channel:

conda update python -c c3i_test2

ccordoba12 commented 5 years ago

Those are great new Ray!! How did you manage to do it?

msarahan commented 5 years ago

https://github.com/AnacondaRecipes/python-feedstock/blob/master/recipe/0020-Add-CondaEcosystemModifyDllSearchPath.patch

mingwandroid commented 5 years ago

Mike beat me to it. Interestingly, numpy had adopted a very similar approach but backed off from doing it.

For sure it's a nuclear option, but the other idea I had is far worse (DLL hooking of kernel32.dll - if that's even possible at present, and if it is, if it would remain possible in the future, and there's also a big question as to whether or not virus scanners would class doing this as a virus).

ccordoba12 commented 5 years ago

Thanks Mike!! It looks very interesting!

So what are you trying to do exactly? Putting Anaconda's Library dir in front of DLL's search path?

mingwandroid commented 5 years ago

We are changing the DLL search order from:

dirname(exe_path)
C:\Windows\System32
PATH

.. to ..

dirname(exe_path)
PATH (which usually ends with C:\Windows\System32 anyway)
CAM-Gerlach commented 5 years ago

@mingwandroid @msarahan Thanks so much! I tested it by removing the .bak extension I'd added to the offending DLL, ensuring the error was triggered with the current Python (3.6.7), and then upgrading to the 3.6.8 build from that channel. Indeed, no error occurred when either trying the aforementioned simplified procedure from the standard Python console that previously produced it, nor launching and running numpy/matplotlib commands from Spyder.

BTW, I got a conda error when trying to switch Python back to the defaults version, but I suspect its unrelated since its happened before several times recently after flipping package versions around. (As far as I know, I had closed Spyder and all Python processes in that environment first, and made sure no Anaconda-related directories were opened in my file manager, upgrading to the latest conda version did not resolve it either, and I couldn't find many relevant search results).

``` Executing transaction: / WARNING conda.gateways.disk:exp_backoff_fn(50): Uncaught backoff with errno EEXIST 17 failed ERROR conda.core.link:_execute(543): An error occurred while uninstalling package 'c3i_test2::python-3.6.8-h9f7ef89_1'. FileExistsError(17, 'Cannot create a file when that file already exists') Attempting to roll back. Rolling back transaction: done [WinError 183] Cannot create a file when that file already exists: 'C:\\Anaconda3\\envs\\spyder-4-lsp\\DLLs\\_decimal.pyd' -> 'C:\\Anaconda3\\envs\\spyder-4-lsp\\DLLs\\_decimal.pyd.c~' ```

I fixed it, as I did in previous cases, by just doing find . -type f -name "*.c~" -delete on the /c/Anaconda3/envs/spyder-4-lsp directory and re-running the install command. For reference, the files found:

``` ./DLLs/select.pyd.c~ ./DLLs/sqlite3.dll.c~ ./DLLs/unicodedata.pyd.c~ ./DLLs/_lzma.pyd.c~ ./DLLs/_multiprocessing.pyd.c~ ./DLLs/_overlapped.pyd.c~ ./DLLs/_socket.pyd.c~ ./DLLs/_sqlite3.pyd.c~ ./DLLs/_ssl.pyd.c~ ./Lib/site-packages/mkl_fft/_pydfti.cp36-win_amd64.pyd.c~ ./Lib/site-packages/numpy/_mklinit.cp36-win_amd64.pyd.c~ ./Library/bin/libifcoremd.dll.c~ ./Library/bin/mkl_rt.dll.c~ ./python3.dll.c~ ./python36.dll.c~ ./pythonw.exe.c~ ```

Thanks!

mingwandroid commented 5 years ago

Thank you for the feedback.

tirthshah147 commented 5 years ago

['System32\libiomp5md.dll', 'System32\mkl_core.dll', 'System32\mkl_def.dll','System32\mkl_intel_thread.dll', 'SysWOW64\libiomp5md.dll','SysWOW64\mkl_core.dll','SysWOW64\mkl_intel_thread.dll', 'SysWOW64\mkl_p4.dll', 'SysWOW64\mkl_p4m.dll','SysWOW64\mkl_p4m3.dll', 'SysWOW64\mkl_p4p.dll']

Remove all these ddl files from C:\Windows\SysWOW64 and C:\Windows\System32

Just do this and restart the spyder once again and hurrrahhhh!!! ur problem will be solved and spyder will open without showing any ordinal 242 error............just do it it is the only solution..

CAM-Gerlach commented 5 years ago

Remove all these ddl files

I strongly advise against the removing the DLLs in case they break something you need (something had to put them there) and instead adding the .bak extension.

C:\Windows\SysWOW64

FYI, this isn't necessary unless for some odd reason you are using the legacy 32-bit version of Anaconda, which you really shouldn't be (unless you're on a >10 year old machine that's unlikely to be able to run Anaconda adequately anyway, or accidentally bought a 32-bit version of Windows that MS keeps around for enterprises running ancient 16-bit DOS software from >3 decades ago.

it it is the only solution..

As you can see in the previous few comments, @mingwandroid stated and I confirmed this should be worked around by the upcoming next release of the Anaconda python interpreter executable, without having to do any of this.

CAM-Gerlach commented 5 years ago

@mingwandroid It seems there is a big discussion going on about this upstream...you involved in that?

msarahan commented 5 years ago

@CAM-Gerlach we're not sure about our new approach with the library loading. It may fix a lot of people, but it may also break a lot more. We're leaning right now towards providing the new changed capability disabled by default, but switchable with an environment variable. I'll link to docs here when we have them.

msarahan commented 5 years ago

PS: I'm having trouble reproducing the error. I would like to try to catch it somehow in order to direct people who encounter the problem to our new docs. Do you have any specific tips on how to reproduce this? I have tried to install ancient MKL into my System32 folder (2018, 2017, even 11), but in a clean python 3.7, numpy 1.15.4, MKL 2019 env, I can't get the error to come up. Procmon shows the old libraries getting loaded from System32 - I'm just confused as to how the error is not showing up. Is there a specific numpy operation that triggers the issue for you?

PS: ancient MKL included intel-openmp (libiomp5md.dll). I have also tried installing older versions there, too.

CAM-Gerlach commented 5 years ago

Thanks for the update @msarahan . Will the docs discuss what to look out for in terms of the types of usecases where enabling this change to the DLL search order for Anaconda would break desired functionality?

msarahan commented 5 years ago

The major one we've found was with pytorch and cuda. https://github.com/pytorch/pytorch/issues/17051

Aside from this, it just scares me to have our software override whatever is in system32. To be sure, the root problem here of 3rd party vendors installing things to system32 is bad, but we should not create potential security/driver confusion by trying to compensate for that.

CAM-Gerlach commented 5 years ago

@msarahan The fastest way I found to reliably trigger it was python then

import matplotlib.pyplot as plt
plt.plot([])

You might need to go older with your DLL versions. I attached the libiomp5md.dll that triggers it for me; its from 2009 vintage.

libiomp5md.dll.bak.zip

CAM-Gerlach commented 5 years ago

Aside from this, it just scares me to have our software override whatever is in system32.

I mean, if I understand correctly its only overriding the DLL loading order for A. Code running on the Anaconda Python interpreter, and B. To only move the local, Anaconda-controlled DLL search directories above System32 (do you check that its somewhere on the PATH already, and if not add it at the end just in case?).

Isn't the worst that can happen (which, isn't good but isn't the end of the world) is it breaks specific packages running under Anaconda that rely on hacky DLL search/load tricks themselves, or for some reason need the version of a DLL in Sysetem32 to appear higher in the path than anything locally?

And in terms of security/driver issues, I assume a package could cause a DoS (or much worse, with some effort) by shipping a bad or malicious ntdll.dll or something, but at that point if it can drop arbitrary DLLs on the system then it could drop any other malicious payload instead anyway, and as far as I understand they are only loaded in the context and permissions of the Python interpreter running the process, which the malicious package presumably already effectively controls?

Or is it possible to drop copies of malicious DLLs locally inside the Anaconda environment on the search path, and then that important DLL get loaded instead of the real one if another Anaconda Python interpreter process performs a low-level system call, in order to intercept data or escalate privileges (e.g. wait for the user in an admin install to run conda as admin, then take control of that process via a malicious low-level DLL dropped on the PATH? I'm no security expert, much less do I have much knowlege about the arcane internals of Windows, but I'm wondering what's really possible or not if this change goes through.

dolson14 commented 5 years ago

@mingwandroid Ray, I may have a use case that isn't solved by the patch.

I recently had to install the Intel Fortran compiler, and even with the c3i_test2 version I have the same dll error.

λ where mkl_core.dll
C:\Program Files x86)\IntelSWTools\compilers_and_libraries_2016.1.146\windows\redist\ia32\mkl\mkl_core.dll
C:\Users\<username>\Programs\Anaconda3\Library\bin\mkl_core.dll

Since the offending dll is not in the System32 directory, is this falling outside of what the patch is trying to accomplish? My initial (albeit uneducated) guess would be that somehow ifort is still taking precedence in the system, but I don't know nearly enough about it to know how to fix that.

CAM-Gerlach commented 5 years ago

@dolson14 Take all of this with a grain of salt since I'm no Windows DLL expert like @mingwandroid and @msarahan , nor am I an Anaconda employee, just a Spyder dev.

However, I'm not sure how the compiler is injecting its location higher onto the path than Anaconda. Assuming conda activate is working correctly, it should always be higher in the search order than anything else, at least when launching anything from the Anaconda prompt, except if the user has run a script/program of some sort after activateing the environment that deliberately set the path differently. In that case, its not really something that can be solved on Anaconda's side in my limited understanding, except by using very techniques like manifests and the like.

Have you run any commands, scripts or programs after activateing your environment? If you type conda activate your-env-name with the latest version of conda, and then run where mkl_core.dll again, do you see the same result? Are you getting the error in Spyder, or something else? What packages have you installed in this environment (conda list --show-channel-urls)? Does it persist in other environments?

msarahan commented 5 years ago

@dolson14 is likely not activating the environment. PATH works in a line. Activating lets us come first without raising hell with other software. Not activating means you're at the mercy of order, where generally only appending is safe, and you're thus in a situation where the earliest installed software has higher priority than newer stuff.