search

ContinuumIO / anaconda-issues

Anaconda issue tracking
648 stars 222 forks source link

Intel MKL FATAL ERROR: Cannot load libmkl_avx.so or libmkl_def.so. #698

Closed kalefranz closed 8 years ago

kalefranz commented 8 years ago

From @jakirkham on February 10, 2016 2:20

Seeing this on Travis CI (Linux) and in Docker containers during install. ( https://travis-ci.org/jakirkham/nanshe/builds/108187039#L749 )

Copied from original issue: conda/conda#2048

kalefranz commented 8 years ago

From @bilderbuchi on February 10, 2016 12:22

I just encountered a similar/equivalent one (on Windows): Intel MKL FATAL ERROR: Cannot load mkl_avx.dll or mkl_def.dll.

kalefranz commented 8 years ago

From @msarahan on February 10, 2016 13:50

Pinging @ilanschnell

kalefranz commented 8 years ago

From @bilderbuchi on February 10, 2016 15:11

Here is another instance of this error, apparently. It provides a repro procedure (which doesn't repro on my side, though), and hints that maybe scipy 0.17 is the culprit, and downgrading to 0.15 could circumvent it?

kalefranz commented 8 years ago

From @jakirkham on February 10, 2016 15:34

FWIW, this problem does not occur on Mac. So, it is only Linux and Windows.

kalefranz commented 8 years ago

From @jakirkham on February 10, 2016 15:39

It appears like the libraries are in the packages.

@bilderbuchi, are using Windows natively or are using it in a VM? If the latter what are you using for virtualization? If the former, do you know what architecture you have?

kalefranz commented 8 years ago

From @bilderbuchi on February 11, 2016 6:59

natively, 64bit, Windows 8.1.

kalefranz commented 8 years ago

From @desilinguist on February 11, 2016 18:15

Yes, I am having the same issue on my RHEL box. Works fine on my Mac. Seems like scipy is the culprit but all the paths seem so the set correctly when I run show_config() and all the libraries seem to be there under $PREFIX/lib.

kalefranz commented 8 years ago

From @jakirkham on February 11, 2016 18:17

Sorry, by architecture, @bilderbuchi, I was meaning what kind of processor are you using?

kalefranz commented 8 years ago

From @bilderbuchi on February 12, 2016 7:45

An Intel(R) Xeon(R) CPU E3-1225 V2 @ 3.20GHz

kalefranz commented 8 years ago

Does this issue still exist? Regardless, going to kick it over to anaconda-issues. Pretty sure it's packaging-related.

jakirkham commented 8 years ago

This was a problem on Linux. I spun up a docker container and installed numpy, scipy, and mkl. I tried importing numpy and scipy, which did not fail indicating this is probably fixed. Also, I tried using numpy.dot, which uses the BLAS if available, and imported/used a few functions from scipy.linalg.blas, which only use the BLAS. These seemed to work and give correct results. So, I believe this is fixed and can be closed.

kalefranz commented 8 years ago

Cool!

KelSolaar commented 8 years ago

I have some Travis builds failing with the same error: https://travis-ci.org/colour-science/colour/builds/118175098

KelSolaar commented 8 years ago

@kalefranz: Is it possible to re-open that issue please, it is not fixed or there are no clear step-by-step info to correct the problem.

jakirkham commented 8 years ago

Why do you need to pin the patch number for NumPy in your builds?

KelSolaar commented 8 years ago

@jakirkham: It is a good point, back then I think we had specific requirements (especially for Scipy). I will try a build without specifying any version. Thanks!

KelSolaar commented 8 years ago

Seems like it was that, for my curiosity any specific reason why this problem happened? Cheers,

jakirkham commented 8 years ago

There were some issues with the first MKL package released as you have seen. In this case, missing libraries on Linux. I expect (though have not checked) that NumPy 1.10.1 was pinned to a certain version of the MKL package. However, this was fixed in a later version of the MKL package and I believe the next NumPy package (think 1.10.2) changed its pinning to this new version.

KelSolaar commented 8 years ago

Excellent, makes sense! Thanks for the help, appreciated :+1:

maximz commented 8 years ago

EDIT: My mistake, the wipe was not as complete as I wanted. With a truly fresh install, this problem is gone.

Leaving the comment below because it shows how to reproduce the original error.

I don't think this is fixed. I am experiencing this issue from a wipe and fresh download and install of Anaconda today. Again, the particular issue is: Intel MKL FATAL ERROR: Cannot load libmkl_avx.so or libmkl_def.so.

Here is a way to reproduce, thanks to https://github.com/BVLC/caffe/issues/3884:

> python -c 'import sklearn.linear_model.tests.test_randomized_l1'
Intel MKL FATAL ERROR: Cannot load libmkl_avx.so or libmkl_def.so.

According to that thread, the solution is to not use MKL or to do the following:

> LD_PRELOAD=~/anaconda2/lib/libmkl_core.so python -c 'import sklearn.linear_model.tests.test_randomized_l1'

However, with LD_PRELOAD set that way, I get this further issue when running my code base (which used to work fine): python: symbol lookup error: /home/username/anaconda2/lib/libmkl_core.so: undefined symbol: mkl_blas_dtrsm

What gives?

> conda list
...
mkl                       11.3.1                        0
mkl-service               1.1.2                    py27_0
...
numpy                     1.10.4                   py27_1
...
scikit-learn              0.17.1              np110py27_0
...
scipy                     0.17.0              np110py27_2
adamdivak commented 8 years ago

I had the same problem, despite using the latest available packages. Turns out the solution was easier than I thought: for whatever reason Anaconda installed the MKL-enabled versions of the numpy/scipy stack, but did not actually install mkl itself. I have seen this when building Docker images based on the Jupyter minimal notebook stack.

A simple conda install --yes mkl mkl-service solved it for me.

ibmua commented 8 years ago

Updating via conda install mkl solved it for me. It seems to have updated several modules including mkl, mkl-service and numpy.

smithsp commented 8 years ago

This is my first post to anaconda-issues, so suggested alternative actions to posting here are welcome.

On my system, I have the following packages installed:

: conda list -n omfit
# packages in environment at /p/omfit/envs/omfit:
#
anaconda-client           1.5.1                    py27_0    defaults
astropy                   1.2.1               np111py27_0    defaults
async                     0.6.1                    py27_0    https://conda.binstar.org/minadyn/linux-64/async-0.6.1-py27_0.tar.bz2
backports                 1.0                      py27_0    defaults
backports.shutil-get-terminal-size 1.0.0                     <pip>
binstar                   0.12                          2    http://repo.continuum.io/pkgs/free/linux-64/binstar-0.12-2.tar.bz2
boto                      2.38.0                    <pip>
boto3                     1.1.2                     <pip>
botocore                  1.1.11                    <pip>
cairo                     1.12.18                       6    defaults
clyent                    1.2.2                    py27_0    defaults
configobj                 5.0.6                    py27_0    http://repo.continuum.io/pkgs/free/linux-64/configobj-5.0.6-py27_0.tar.bz2
curl                      7.49.0                        1    defaults
cvxopt                    1.1.8                    py27_3    defaults
cycler                    0.10.0                   py27_0    defaults
cython                    0.24.1                   py27_0    defaults
dateutil                  2.4.1                    py27_0    http://repo.continuum.io/pkgs/free/linux-64/dateutil-2.4.1-py27_0.tar.bz2
decorator                 4.0.10                   py27_0    defaults
docutils                  0.12                      <pip>
emcee                     2.1.0                     <pip>
fontconfig                2.11.1                        6    defaults
fortranformat             0.2.5                    py27_0    https://conda.binstar.org/auto/linux-64/fortranformat-0.2.5-py27_0.tar.bz2
freetype                  2.5.5                         1    defaults
futures                   2.2.0                     <pip>
get_terminal_size         1.0.0                    py27_0    defaults
gitdb                     0.5.4                    py27_0    https://conda.binstar.org/minadyn/linux-64/gitdb-0.5.4-py27_0.tar.bz2
gitpython                 0.3.2.RC1                py27_0    https://conda.binstar.org/minadyn/linux-64/gitpython-0.3.2.RC1-py27_0.tar.bz2
glib                      2.43.0                        1    defaults
gptools                   0.2.2                     <pip>
harfbuzz                  0.9.39                        1    defaults
hdf4                      4.2.12                        0    defaults
hdf5                      1.8.17                        1    defaults
ipython                   5.0.0                    py27_0    defaults
ipython-genutils          0.1.0                     <pip>
ipython_genutils          0.1.0                    py27_0    http://repo.continuum.io/pkgs/free/linux-64/ipython_genutils-0.1.0-py27_0.tar.bz2
jbig                      2.1                           0    defaults
jmespath                  0.7.1                     <pip>
jpeg                      8d                            1    defaults
lcms                      1.19                          0    http://repo.continuum.io/pkgs/free/linux-64/lcms-1.19-0.tar.bz2
libffi                    3.2.1                         0    defaults
libgfortran               3.0.0                         1    defaults
libnetcdf                 4.4.1                         0    defaults
libpng                    1.6.22                        0    defaults
libtiff                   4.0.6                         2    defaults
libxml2                   2.9.2                         0    http://repo.continuum.io/pkgs/free/linux-64/libxml2-2.9.2-0.tar.bz2
lmfit                     0.9.2                     <pip>
matplotlib                1.5.1               np111py27_0    defaults
mdsplus-6.1               82                        <pip>
mkl                       11.3.3                        0    defaults
mkl-service               1.1.2                    py27_2    defaults
mpmath                    0.19                      <pip>
netcdf4                   1.2.4               np111py27_0    defaults
numeric                   24.2                      <pip>
numpy                     1.11.1                   py27_0    defaults
openblas                  0.2.14                        4    defaults
openssl                   1.0.2h                        1    defaults
pandas                    0.17.1                    <pip>
pango                     1.39.0                        1    defaults
path.py                   8.2.1                    py27_0    defaults
pathlib2                  2.1.0                    py27_0    defaults
pexpect                   4.0.1                    py27_0    defaults
pickleshare               0.7.3                    py27_0    defaults
pidly                     0.2.6                     <pip>
pil                       1.1.7                    py27_2    defaults
pillow                    3.3.0                    py27_0    defaults
pip                       8.1.2                    py27_0    defaults
pixman                    0.32.6                        0    defaults
prompt-toolkit            1.0.3                     <pip>
prompt_toolkit            1.0.3                    py27_0    defaults
ptyprocess                0.5.1                    py27_0    defaults
py2cairo                  1.10.0                   py27_2    http://repo.continuum.io/pkgs/free/linux-64/py2cairo-1.10.0-py27_2.tar.bz2
pycairo                   1.10.0                   py27_0    defaults
pygments                  2.1.3                    py27_0    defaults
pyodbc                    3.0.10                   py27_0    defaults
pyparsing                 2.1.4                    py27_0    defaults
pyqt                      4.11.4                   py27_4    defaults
python                    2.7.12                        1    defaults
python-dateutil           2.5.3                    py27_0    defaults
pytz                      2016.6.1                 py27_0    defaults
pyyaml                    3.11                     py27_4    defaults
qt                        4.8.7                         4    defaults
readline                  6.2                           2    http://repo.continuum.io/pkgs/free/linux-64/readline-6.2-2.tar.bz2
requests                  2.10.0                   py27_0    defaults
scipy                     0.18.0              np111py27_0    defaults
setuptools                25.1.6                   py27_0    defaults
simplegeneric             0.8.1                    py27_1    defaults
sip                       4.18                     py27_0    defaults
six                       1.10.0                   py27_0    http://repo.continuum.io/pkgs/free/linux-64/six-1.10.0-py27_0.tar.bz2
smmap                     0.8.2                    py27_0    https://conda.binstar.org/minadyn/linux-64/smmap-0.8.2-py27_0.tar.bz2
sqlite                    3.13.0                        0    defaults
system                    5.8                           2    http://repo.continuum.io/pkgs/free/linux-64/system-5.8-2.tar.bz2
tk                        8.5.18                        0    http://repo.continuum.io/pkgs/free/linux-64/tk-8.5.18-0.tar.bz2
traitlets                 4.2.2                    py27_0    defaults
triangle                  2015.12.14                <pip>
triangle-plot             0.3.0                     <pip>
uncertainties             2.4.4                    py27_0    https://conda.binstar.org/auto/linux-64/uncertainties-2.4.4-py27_0.tar.bz2
unixodbc                  2.3.4                         0    defaults
wcwidth                   0.1.7                    py27_0    defaults
wheel                     0.29.0                   py27_0    defaults
xarray                    0.7.0                     <pip>
xz                        5.2.2                         0    defaults
yaml                      0.1.6                         0    http://repo.continuum.io/pkgs/free/linux-64/yaml-0.1.6-0.tar.bz2
zlib                      1.2.8                         3    defaults

That is after I did conda update --all -n omfit and conda install -f -n omfit mkl mkl-service. However, I am still seeing the following problem:

: python
Python 2.7.12 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:42:40) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> from cvxopt import solvers
>>> from cvxopt import matrix
>>> 
>>> Q = 2*matrix([ [2., .5],[.5, 1.] ] )
Intel MKL FATAL ERROR: Cannot load libmkl_def.so.

Is there any workaround or suggested action?

jskDr commented 8 years ago

I got also the following error: python: symbol lookup error: /home/username/anaconda2/lib/libmkl_core.so: undefined symbol: mkl_blas_dtrsm

jskDr commented 8 years ago

I finally solved this problem using two steps for my deep learning applications with Keras/Theano. Notice that I am using Ubuntu 14.04.

First, I removed mkl with the following two commands. $ [sudo] conda install nomkl numpy scipy scikit-learn numexpr $ [sudo] conda remove mkl mkl-service

Although mkl is removed from my anaconda python 3, LinearRegression fit in skearn still makes error related to scipy. During I am searching Web. Some brilliants said to remove python-scipy but install pip based way. So, I applied this solution to my case. I removed conda scipy and install pip sciy py as follows: $ [sudo] conda remove scipy $ [sudo] pip install scipy while scipy was removed from conda, it also took his followers such as sklearn. Hence, I installed sklearn again using the pip tool such as. $ [sudo] pip install sklearn

Now everything works perfectly. I am very fine without invoking mkl.

smithsp commented 8 years ago

Following the instructions of @jskDr worked for me.
Thanks @jskDr . 👍

maksimt commented 8 years ago

Also had to use @jskDr 's solution on a fresh anaconda install on Ubuntu 16.04. The other upvoted methods didn't work for me. I'm hoping to be able to use MKL in the future.

nehaljwani commented 7 years ago

I came across the same error while building cvxopt with mkl:11.3.3 and mkl:2017.0.0, separately. I got around it by linking libmkl_rt.so and now no LD_PRELOAD trick is required. The linking order used by me was: -lmkl_rt -lmkl_core -lmkl_intel_lp64 -lmkl_sequential

oxydron commented 7 years ago

@jskDr solution indeed do work and don't know why.

Thanks mate!

rcywongaa commented 7 years ago

Occurs when using python3.5 but not python3.6

jdhao commented 7 years ago

I am using Anaconda 2 on Centos 7, I also met this issue. Following @jskDr 's solution, the problem disappeared. Thanks!

chrishyx commented 7 years ago

I am using Anaconda3-4.3.0 on Ubuntu16.04, also met this issue and solved by following @jskDr 's solution. Thanks!

maartenbreddels commented 7 years ago

For me the problem was resolved by removing numpy from ~/.local/.... If any pip install --user <package> command dragged numpy to install in ~/.local/lib/pythonX.Y/site-packages/numpy/ it will always use that numpy, and conda list will not show it, so you may thing you don't have that installed. pip uninstall numpy fixed it. I think that explains many of the issues, for instance switching to 3.6, since numpy for 3.5 won't be found anymore. Possibly similar issues exists for scipy, I didn't check.

havakv commented 7 years ago

For me (on Ubuntu 14.04, python 3.6.1) it started working after installing numexpr (conda install numexpr). No paths specified.

MInner commented 7 years ago

For me problems started appearing after installing a Python 2.7 with corresponding numpy alongside root env. Creating a new env with python 3 and numpy helped and none of suggestions above did not.

pamdla commented 7 years ago

This issue still exists.

My environment configs are: VM: {gues_os: Ubutu 14.04 64, anaconda2: 4.4.0} And I can assure both mkl and mkl-service are installed, and resides at '/opt/anaconda2/lib/'. However, when I fire python -c "import gensim", the exact error msg is: Intel MKL FATAL ERROR: Cannot load libmkl_avx.so or libmkl_def.so

Using LD_PRELOAD=... does not help me out either. Using LD_DEBUG=symbol python -c 'import gensim' does give me details, and found 4 places for ****

1.  9582:   symbol=mkl_dft_avx_xs_f32_1df;  lookup in file=/opt/anaconda2/lib/python2.7/site-packages/scipy/special/../../../../libmkl_avx.so [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=python [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/opt/anaconda2/bin/../lib/libpython2.7.so.1.0 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib/x86_64-linux-gnu/libpthread.so.0 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib/x86_64-linux-gnu/libdl.so.2 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib/x86_64-linux-gnu/libutil.so.1 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib/x86_64-linux-gnu/libm.so.6 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/opt/anaconda2/lib/python2.7/site-packages/scipy/special/../../../../libmkl_avx.so [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib/x86_64-linux-gnu/libdl.so.2 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
      **9582:   /opt/anaconda2/lib/python2.7/site-packages/scipy/special/../../../../libmkl_avx.so**: error: symbol lookup error: undefined symbol: mkl_sparse_optimize_bsr_trsm_i8 (fatal)

2. 9582:    symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/opt/anaconda2/bin/../lib/libpython2.7.so.1.0 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib/x86_64-linux-gnu/libpthread.so.0 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib/x86_64-linux-gnu/libdl.so.2 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib/x86_64-linux-gnu/libutil.so.1 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib/x86_64-linux-gnu/libm.so.6 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/opt/anaconda2/lib/python2.7/site-packages/scipy/special/../../../../libmkl_def.so [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib/x86_64-linux-gnu/libdl.so.2 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
      9582: symbol=mkl_sparse_optimize_bsr_trsm_i8;  lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
     ** 9582:   /opt/anaconda2/lib/python2.7/site-packages/scipy/special/../../../../libmkl_def.so**: error: symbol lookup error: undefined symbol: mkl_sparse_optimize_bsr_trsm_i8 (fatal)

It reveals that the installed scipy is referencing MKL libs. But the installed MKL coming along with anaconda is located at '/opt/anaconda2/lib/libmkl_{core,avx,def}.so'.

And for aforementioned solutions, I believe either to uninstall mkl or to install nomkl is not solving this issue. If understand correctly, nomkl is NON-MKL version for originally mkl-relavant packages such as scipy, numpy. Therefore installing nomkl will not give us MKL's benefit for later use while mkl- relevant boosting was assumed in MKL-verion of numpy/scipy/etc, right?

I tried to find which file is referencing these so file with grep -r lmkl_avx and grep -r lmkl_def, but no result returned.

I also installed Intel MKL package downloaded from Intel's site. Similar error log apears, but in addition I found:

      9517: symbol=COIProcessLoadSinkLibraryFromFile;  lookup in file=/opt/anaconda2/lib/python2.7/site-packages/scipy/special/../../../../libmkl_intel_lp64.so [0]
      9517: symbol=COIProcessLoadSinkLibraryFromFile;  lookup in file=/opt/anaconda2/lib/python2.7/site-packages/scipy/special/../../../../libmkl_intel_thread.so [0]
      9517: symbol=COIProcessLoadSinkLibraryFromFile;  lookup in file=/opt/anaconda2/lib/python2.7/site-packages/scipy/special/../../../../libmkl_core.so [0]
      9517: symbol=COIProcessLoadSinkLibraryFromFile;  lookup in file=/opt/anaconda2/lib/python2.7/site-packages/scipy/special/../../../../libiomp5.so [0]
     ........

The problem, I guess, is how to correctly link the /opt/anaconda2/lib/liblmk_{core|avx|def}.so for scipy or gensim, by hard code or dynamic way -ld.

I will continuing working and the post a solution, if no one solves.

pamdla commented 7 years ago

In my problematic Ubuntu vbox, this issue is solved by using:

LD_PRELOAD=/opt/anaconda2/lib/libmkl_core.so:/opt/anaconda2/lib/libmkl_sequential.so python -c 'import gensim'

Also, in another copy of Ubuntu vbox as well as Centos vbox, I can do python -c "import gensim" after installing the same version (4.4.0) of anaconda and gensim in my problematic Ubuntu without any additional settings.

It seems that the issue has sth to with OS or Shell environment.

mrdivine commented 7 years ago

On my scientific linux box, I fixed this issue by pinning mkl to version 11.3.3

conda install mkl=11.3.3

akshay-hazari commented 7 years ago

Nothing seems to work for me here. Though for me its not an anaconda issiue may be. I am getting this error from a VirtualEnv set up for django. Any help is appreciated,

akshay-hazari commented 7 years ago

This did the trick when everything else failed - sudo cp /home/akshay/anaconda2/pkgs/mkl-2017.0.1-0/lib/* /usr/lib Or this might also work export PATH="/home/akshay/anaconda2/pkgs/mkl-2017.0.1-0/lib:$PATH" While someone else also suggested this export PATH="/home/anaconda2/bin:$PATH"

emailweixu commented 7 years ago

Executing the following before other imports solves the problem for me: sys.setdlopenflags(sys.getdlopenflags() | ctypes.RTLD_GLOBAL)

rprasadn commented 6 years ago

I was having this error inside a conda environment using py3.5 in Ubuntu 14.04. My problem was resolved by executing conda install mkl inside the environment.

Nzumino commented 6 years ago

I had the "Intel MKL FATAL ERROR: Error on loading function mkl_blas_avx_xdcopy". It was resolved by executing conda update mkl.

SankBad commented 6 years ago

This issue is still persisting for me

2020zyc commented 2 years ago

The above solutions are invalid for me, and I solved it by adding the two lines to the beginning of my code: """ import mkl mkl.get_max_threads() """