search

haasad / PyPardiso

Python interface to the Intel MKL Pardiso library to solve large sparse linear systems of equations
BSD 3-Clause "New" or "Revised" License
133 stars 20 forks source link

pypardiso will not build out of pypi on LINUX cluster #54

Closed jhein32 closed 11 months ago

jhein32 commented 11 months ago

I am having problems building pypardiso on our LINUX cluster using pypi:

[jhein@cosmos-sw ~]$ pip install --prefix=$HOME/test_paradiso pypardiso
Collecting pypardiso
  Using cached pypardiso-0.4.2-py3-none-any.whl (10 kB)
Requirement already satisfied: scipy in /sw/easybuild_milan/software/SciPy-bundle/2022.05-foss-2022a/lib/python3.10/site-packages (from pypardiso) (1.8.1)
Collecting mkl
  Using cached mkl-2023.2.0-py2.py3-none-manylinux1_x86_64.whl (258.0 MB)
Requirement already satisfied: numpy in /sw/easybuild_milan/software/SciPy-bundle/2022.05-foss-2022a/lib/python3.10/site-packages (from pypardiso) (1.22.3)
Collecting tbb==2021.*
  Using cached tbb-2021.10.0-py2.py3-none-manylinux1_x86_64.whl (4.4 MB)
Collecting intel-openmp==2023.*
  Using cached intel_openmp-2023.2.0-py2.py3-none-manylinux1_x86_64.whl (20.2 MB)
Installing collected packages: tbb, intel-openmp, mkl, pypardiso
Successfully installed intel-openmp-2023.2.0 mkl-2023.2.0 pypardiso-0.4.2 tbb-2021.10.0

So far so good. When I try to run it, I get 

[jhein@cosmos-sw ~]$ python
Python 3.10.4 (main, Jan 30 2023, 12:07:46) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pypardiso
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jhein/test_paradiso/lib/python3.10/site-packages/pypardiso/__init__.py", line 4, in <module>
    from .scipy_aliases import spsolve, factorized
  File "/home/jhein/test_paradiso/lib/python3.10/site-packages/pypardiso/scipy_aliases.py", line 9, in <module>
    pypardiso_solver = PyPardisoSolver()
  File "/home/jhein/test_paradiso/lib/python3.10/site-packages/pypardiso/pardiso_wrapper.py", line 93, in __init__
    raise ImportError('Shared library mkl_rt not found')
ImportError: Shared library mkl_rt not found

The issue is that the pypi installed mkl supplies a libmkl_rt.so.2 but not a libmkl_rt.so. If add another mkl library to the path which supplies a libmkl_rt.so`, the module will load.

Could you please comment?

haasad commented 11 months ago

There was a similar issue in the past on windows: https://github.com/haasad/PyPardisoProject/issues/12

I'll check and get back to you

haasad commented 11 months ago

As far as I can tell the reason the mkl_rt library isn't discovered is because you use the --prefix option with pip install.

I'd recommend to use a virtual environment instead to install pypardiso (as is also suggested in the README). Something like this should do the trick:

cd $HOME/test_paradiso
python -m venv pypardiso_venv
source pypardiso_venv/bin/activate
pip install pypardiso

see https://docs.python.org/3.10/library/venv.html

haasad commented 11 months ago

If you don't want to use a virtual environment, pip install --user pypardiso should also work.

haasad commented 11 months ago

~Or if you just want to fix it for your current installation, creating a symbolic link will work:~

cd $HOME/test_paradiso/lib
ln -s libmkl_rt.so.2 libmkl_rt.so
haasad commented 11 months ago

I'm closing this issue, feel free to reopen it if my suggestions didn't fix your problem.