ImportError: No module named multiarray

[ikrambennasrbennasr]

Hello , please i run this code :

print the summary of the architecture

model.summary()

load the loss summary (training and validation losses)

import pickle losses = pickle.load( open('./tmpdata/history_exp' + str(exp_number) + '.pickle' , 'rb'))

load the test set

test_set = pickle.load( open('./tmpdata/TestSet.pickle', 'rb')) X_test = test_set['X_test'] Y_test = test_set["Y_test"]

and i get this issue :

ImportError Traceback (most recent call last)

in () 5 exp_number = 15 6 with open('/home/ubuntu/nilmtk/nilmtk/neuralnilm/tmpdata/history_exp' + str(exp_number) + '.pickle' , 'rb') as f: ----> 7 losses = pickle.load(f) 8 # load the test set 9 with open('/home/ubuntu/nilmtk/nilmtk/neuralnilm/tmpdata/TestSet.pickle', 'rb') as v: ImportError: No module named multiarray can someone please help me

[mattip]

You may need to do from numpy.core import multiarray before pickle.load

[ikrambennasrbennasr]

@mattip Thanks a lot for the response but when i do it :

exp_number = 13 model.summary() from numpy.core import multiarray import cPickle as pickle losses = pickle.load( open('/home/ubuntu/nilmtk/nilmtk/neuralnilm/tmpdata/history_exp' + str(exp_number) + '.pickle' , 'rb')) test_set = pickle.load( open('/home/ubuntu/nilmtk/nilmtk/neuralnilm/tmpdata/TestSet.pickle', 'rb')) X_test = test_set['X_test'] Y_test = test_set["Y_test"]

the error persists :

in () 7 losses = pickle.load( open('/home/ubuntu/nilmtk/nilmtk/neuralnilm/tmpdata/history_exp' + str(exp_number) + '.pickle' , 'rb')) 8 # load the test set ----> 9 test_set = pickle.load( open('/home/ubuntu/nilmtk/nilmtk/neuralnilm/tmpdata/TestSet.pickle', 'rb')) 10 X_test = test_set['X_test'] 11 Y_test = test_set["Y_test"] ImportError: No module named multiarray

[mattip]

Perhaps a duplicate of #1882, in that one whoever created the pickle file should use 'wb'

[ikrambennasrbennasr]

I have already changed and the error persists :(

[a093130]

Me too. Using Goddard Mission Analysis Toolkit. Trying to run a Python user function (Python 3.6.4).

"Error in the CallPython command execution: Interface Exception Thrown: Python Exception: <class 'ImportError'>: Importing the multiarray numpy extension module failed. ... Original error was: DLL load failed: The specified module could not be found."

Why does mattip think this is invalid? Looks like new behavior.

[matthew-brett]

@a093130 - does your stacktrace also show you are trying to load a pickle file? If not, what you are seeing is a very general error when your numpy install is broken. I don't know the Goddard Mission Analysis Toolkit, but does it ship with a version of numpy? Is it their numpy that is broken? If so, can you contact them with this report?

[ikrambennasrbennasr]

Hi , i try this code and it runs : with open('/home/ubuntu/nilmtk/nilmtk/tmpdata/TestSet.pickle', 'rb') as fid: u = pickle._Unpickler(fid) u.encoding = 'latin1' test_set = u.load()

[a093130]

GMAT does not ship with a version of Numpy, but they do start their own bundled install of the Python 3.6.4 interpreter. The funny is that my Python installation is under Anaconda and I used conda install numpy to install it. Given this error I updated numpy under the Anaconda environment. I can "import numpy.core.multiarray" under Spyder in the Anaconda build, can use numpy and scipy freely in my standalone code. It seems maybe I need to go to the GMAT bug tracker. Thanks for the reply.

[mattip]

@a093130, @ikrambennasrbennasr did you work out what is going on?

[a093130]

I did,
This issue seems to be common to Windows with any Python installation. Several issues are posted in numpy github.

Issue 9272, has a thread with a workaround at the end , very long discussion, but I'll cut to the end:

@pfmoore commented on Jun 21, 2017 ... now the problem is, that there are multiple versions for each of the DLLs (DLL hell) all different dates, versions, file sizes. Basically the Anaconda Navigator or Spyder or python installed through different environments do not load the correct DLL versions. This is an issue with numpy loading old versions from the C:\Windows\System32\ folder.

mkl_intel_thread.dll C:\Anaconda3\Library\bin\mkl_intel_thread.dll (october 2016) C:\Anaconda3\pkgs\mkl-2017.0.3-0\Library\bin\mkl_intel_thread.dll (april 2017) C:\Windows\System32\mkl_intel_thread.dll (v2009) mkl_core.dll C:\Anaconda3\pkgs\mkl-2017.0.3-0\Library\bin\mkl_core.dll (april 2017) C:\Anaconda3\Library\bin\mkl_core.dll (october 2016) C:\Windows\System32\mkl_core.dll (v2009) libiomp5md.dll C:\Anaconda3\pkgs\mkl-2017.0.3-0\Library\bin\libiomp5md.dll C:\Anaconda3\Library\bin\libiomp5md.dll C:\Anaconda3\envs\tensorflow-cpu\Library\bin\libiomp5md.dll C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\ia32\compiler\libiomp5md.dll

C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64\compiler\libiomp5md.dll C:\Windows\System32\libiomp5md.dll

@njsmith commented on Jul 23, 2017. @tobigithub: if you're having trouble loading MKL libraries, then you should take that to the anaconda developers, the numpy project doesn't distribute any binaries that use MKL, and conda has its own ways of trying to avoid DLL hell that we can't really help you with.

@njsmith and @carlkl thank you. There are literally hundreds of applications that use these intel libraries (including Microsoft and Intel itself). With thousands of applications installed sure something will break, but updating older system DLLS (2009) if they predate an new issue will probably create a disaster. It might be system critical such as an INTEL RAID driver or USB or graphics adapter will fail. Just reporting this issue here, its just a hunch, numpy 1.13.1 just worked fine before and now in July 2017 it is magically broken, this may be related to software installs, windows updates or any other packages or events, the error message itself probably also does not help.

Similar issues with numpy and DLLs: https://www.google.com/search?q=numpy+"Importing+the+multiarray+numpy+extension+module+failed."&oq=numpy+"Importing+the+multiarray+numpy+extension+module+failed."

tobigithub commented on Jul 23, 2017. @njsmith thank you, I just confirmed using a VM on a clean Windows 10 system that has never seen Intel MKL DLLs, numpy or conda, all things are fine and numpy works. I am just puzzled how numpy suddenly stopped working, could be third parties. Probably needs some more digging, lets see what we can find out to make numpy ironclad to any dependency issue.

...ok here the solution

The solution is indeed related to some DLL depencency issues under WIN with Intel MKL libraries the numpy import error was under python 3.6.2 and numpy 1.13.1 :

"import numpy" numpy\coreinit.py", line 16, in from . import multiarray ImportError: DLL load failed: The specified procedure could not be found.

The quick and dirty solution is to copy all Intel mkl*.dll libraries from the specific environments (env) subfolder (seen at stackoverflow)

<\Library\bin> to <\Lib\site-packages\numpy\core>

04/13/2017 12:50 AM 30,277,392 mkl_avx.dll 04/13/2017 12:50 AM 38,253,840 mkl_avx2.dll 04/13/2017 12:50 AM 42,234,640 mkl_avx512.dll 04/13/2017 12:50 AM 45,722,896 mkl_avx512_mic.dll 04/13/2017 12:50 AM 25,369,360 mkl_core.dll 04/13/2017 12:50 AM 24,541,456 mkl_def.dll 04/13/2017 12:50 AM 24,506,640 mkl_intel_thread.dll 04/13/2017 12:50 AM 29,744,400 mkl_mc.dll 04/13/2017 12:50 AM 30,594,320 mkl_mc3.dll 04/13/2017 12:50 AM 12,567,824 mkl_rt.dll 04/13/2017 12:50 AM 11,388,688 mkl_sequential.dll 04/13/2017 12:50 AM 12,575,504 mkl_tbb_thread.dll 04/13/2017 12:50 AM 11,250,960 mkl_vml_avx.dll 04/13/2017 12:50 AM 11,463,952 mkl_vml_avx2.dll 04/13/2017 12:50 AM 10,285,840 mkl_vml_avx512.dll 04/13/2017 12:50 AM 11,692,304 mkl_vml_avx512_mic.dll 04/13/2017 12:50 AM 5,224,208 mkl_vml_cmpt.dll 04/13/2017 12:50 AM 5,345,552 mkl_vml_def.dll 04/13/2017 12:50 AM 9,477,904 mkl_vml_mc.dll 04/13/2017 12:50 AM 9,368,336 mkl_vml_mc2.dll 04/13/2017 12:50 AM 9,593,616 mkl_vml_mc3.dll now numpy can be imported and runs fine.

[a093130]

Matti,

Yes, I found another similar issue with a successful workaround. The underlying problem with Anaconda and the location of its mkl libraries – I don’t know if that has been fixed.

Reference Issue 9272, from pfmoore Jun 20, 2017. Applicable thread follows.

pfmoore commented on Jun 21, 2017.

I can confirm that upgrading to 3.6.1 fixes my issue on the machine I originally had the problem with. Personally, I'm OK for this issue to be closed with "upgrade to 3.6.1" as the resolution, but I'll leave it to you to close it in case you'd prefer to fix the binary to support 3.6.0.

tobigithub commented on Jul 22, 2017.

ame issue, numpy v1.13.1 and and python 3.6.1 worked for couple of months, now it just fails. This was based on a conda installation using the windows anaconda navigator. May and June 2017 all code runs fine, then July with an update and and python 3.6.2 "import numpy" does not work anymore.

Some further investigation (only based on numpy and python 3.6.2 leads to

C:\Anaconda3\envs\tensorflow-cpu\Lib\site-packages\numpy\core\multiarray.cp36-win_amd64.pyd

(compiled python DLL) from 7/7/2017 which loads

C:\Windows\System32\mkl_intel_thread.dll (v2009)

C:\Windows\System32\mkl_core.dll (v2009)

C:\Windows\System32\libiomp5md.dll (v2009)

now the problem is, that there are multiple versions for each of the DLLs (DLL hell)

all different dates, versions, file sizes.

Basically the Anaconda Navigator or Spyder or python installed through different environments do not load the correct DLL versions.

This is an issue with numpy loading old versions from the C:\Windows\System32\ folder.

mkl_intel_thread.dll

C:\Anaconda3\Library\bin\mkl_intel_thread.dll (october 2016)

C:\Anaconda3\pkgs\mkl-2017.0.3-0\Library\bin\mkl_intel_thread.dll (april 2017)

C:\Windows\System32\mkl_intel_thread.dll (v2009)

mkl_core.dll

C:\Anaconda3\pkgs\mkl-2017.0.3-0\Library\bin\mkl_core.dll (april 2017)

C:\Anaconda3\Library\bin\mkl_core.dll (october 2016)

C:\Windows\System32\mkl_core.dll (v2009)

libiomp5md.dll

C:\Anaconda3\pkgs\mkl-2017.0.3-0\Library\bin\libiomp5md.dll

C:\Anaconda3\Library\bin\libiomp5md.dll

C:\Anaconda3\envs\tensorflow-cpu\Library\bin\libiomp5md.dll

C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\ia32\compiler\libiomp5md.dll

C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64\compiler\libiomp5md.dll

C:\Windows\System32\libiomp5md.dll

…

The quick and dirty solution is to copy all Intel mkl*.dll libraries from the specific environments (env) subfolder (seen at stackoverflow)

<\Library\bin> to

<\Lib\site-packages\numpy\core>

04/13/2017 12:50 AM 30,277,392 mkl_avx.dll

04/13/2017 12:50 AM 38,253,840 mkl_avx2.dll

04/13/2017 12:50 AM 42,234,640 mkl_avx512.dll

04/13/2017 12:50 AM 45,722,896 mkl_avx512_mic.dll

04/13/2017 12:50 AM 25,369,360 mkl_core.dll

04/13/2017 12:50 AM 24,541,456 mkl_def.dll

04/13/2017 12:50 AM 24,506,640 mkl_intel_thread.dll

04/13/2017 12:50 AM 29,744,400 mkl_mc.dll

04/13/2017 12:50 AM 30,594,320 mkl_mc3.dll

04/13/2017 12:50 AM 12,567,824 mkl_rt.dll

04/13/2017 12:50 AM 11,388,688 mkl_sequential.dll

04/13/2017 12:50 AM 12,575,504 mkl_tbb_thread.dll

04/13/2017 12:50 AM 11,250,960 mkl_vml_avx.dll

04/13/2017 12:50 AM 11,463,952 mkl_vml_avx2.dll

04/13/2017 12:50 AM 10,285,840 mkl_vml_avx512.dll

04/13/2017 12:50 AM 11,692,304 mkl_vml_avx512_mic.dll

04/13/2017 12:50 AM 5,224,208 mkl_vml_cmpt.dll

04/13/2017 12:50 AM 5,345,552 mkl_vml_def.dll

04/13/2017 12:50 AM 9,477,904 mkl_vml_mc.dll

04/13/2017 12:50 AM 9,368,336 mkl_vml_mc2.dll

04/13/2017 12:50 AM 9,593,616 mkl_vml_mc3.dll

now numpy can be imported and runs fine.

From: Matti Picus notifications@github.com Sent: Tuesday, August 7, 2018 5:22 PM To: numpy/numpy numpy@noreply.github.com Cc: Colin chelms@socal.rr.com; Mention mention@noreply.github.com Subject: Re: [numpy/numpy] ImportError: No module named multiarray (#10981)

@a093130 https://github.com/a093130 , @ikrambennasrbennasr https://github.com/ikrambennasrbennasr did you work out what is going on?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/numpy/numpy/issues/10981#issuecomment-411243863 , or mute the thread https://github.com/notifications/unsubscribe-auth/ATd63KKFOF9fJ-i_xk0p8_N3u4c32KLYks5uOi8vgaJpZM4TlCVi . https://github.com/notifications/beacon/ATd63Mc-J2L0Ah16MJKSKKbjdXtL5Venks5uOi8vgaJpZM4TlCVi.gif

[mattip]

Thanks for the followup, glad you worked it out. Note that the original issue occurred after from numpy.core import multiarray succeeded, so leaving this open for now.

[tobigithub]

@a093130 "if you're having trouble loading MKL libraries, then you should take that to the anaconda developers, the numpy project doesn't distribute any binaries that use MKL, and conda has its own ways of trying to avoid DLL hell that we can't really help you with."

Agree, thanks for looking into it, I actually reported that to Continuum Anaconda, but the (real) issue still exists. https://github.com/ContinuumIO/anaconda-issues/issues/3463

I removed the Intel MKLs because of problems, with numpy and TF and keras dependency issues. Not sure if OpenBlas would be a solution, but it probably would be overkill to include fast low level functions into numpy, especially with CPU architectures changing so quickly. But Intel MKL is fast: https://software.intel.com/en-us/articles/performance-comparison-of-openblas-and-intel-math-kernel-library-in-r

[kumar-anil-telikicherla]

It was indeed a problem Anaconda numpy that is not built with mkl libraries. I found a simple fix by replacing the Anaconda numpy installation with an MKL built version:

1. Download the latest numpy+mkl wheel file from https://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy for the python version and system architecture, in my case: numpy‑1.15.3+mkl‑cp37‑cp37m‑win_amd64.whl
2. Open Anaconda prompt and type: pip install ..\numpy‑1.15.3+mkl‑cp37‑cp37m‑win_amd64.whl (full path to the downloaded wheel file)

[mattip]

Glad you figured it out. The solution seems like it would break more than it solves, but whatever works for you. Closing.

[forex4pty]

I had the same Issue, I tried everything I found on the web, yesterday my python was working flawlessly 3.7.1 with numpy 1.15.4, uninstall everything python, numpy several times, and the error of the multiarray was still there, so I copy that file shown on the screenshot is a dll found on /numpy/.Libs is the only file in there, I tried to make a path to it without success, so my last attempt was to copy it directly to /numpy folder and /numpy/core the 2nd one was the one that fixed my issue, hope anyone with the same problem get thru it, I lost my whole morning trying to fix this problem

[ssrokyz]

In the case of Linux Centos6 & 7 (my case) Simply just using python compiled with Intel compilers instead of gnu compilers fixed problem. I think that installing python with intel compilers include mkl libraries in default.

[a093130]

Numpy version 1.12 included a fix for the MKL library problem, which has been over a year. I encountered a recurrence of the problem again (on Win 10) when I made a distribution of my library developed under Anaconda3 which I had straightened out with my manual workaround above, but installed my library in a "vanilla" Python 3.6 environment for use with Visual Studio (note this is the compiler I use). I simply rebuilt numpy at 1.16.1 stable release, installed it, and it ran mklinit.py during the installation and my library executes properly. The mklinit.py script seems to be a new addition.