haasad / PyPardiso

Python interface to the Intel MKL Pardiso library to solve large sparse linear systems of equations
BSD 3-Clause "New" or "Revised" License
136 stars 20 forks source link

Solver failed with code -2 when running in jupyter notebook #38

Closed shushu-qin closed 1 year ago

shushu-qin commented 2 years ago

Hi, I am using 0.4.0 version of the package. I tried the simple example in a python script file. It can be run without any error. However, if I run the same code in a jupyter notebook or use pypardiso in a member function of a class, it fails. Can anyone help resolve this issue?

The code I was testing is as follows: ` import pypardiso import numpy as np import scipy.sparse as sp from scipy.io import loadmat

A = sp.rand(10, 10, density=0.5, format='csr') b = np.random.rand(10) x = pypardiso.spsolve(A, b) print(x) `

And the error I received is: Screenshot from 2022-03-24 15-10-44

haasad commented 2 years ago

Hi @shushu-qin,

I tried to reproduce your issue:

python3 -m venv venv
. ./venv/bin/activate
pip install pypardiso jupyter
jupyter notebook

image

Can you give me some more details? What OS are you using? What's the output of pip list of your virtualenv?

Error code -2 of the Pardiso solver is short for not enough memory. Is there some memory restriction in place on your system that could interfere with the jupyter kernel?

shushu-qin commented 2 years ago

Thank you for the reply. I am gonna give as much information as possible 😄

  1. The OS is Ubuntu 20.04.2 LTS.
  2. Here is the output of pip list pip_list.txt
  3. CPU: 12th Gen Intel(R) Core(TM) i9-12900KF with configuration: cores=16 enabledcores=16 threads=24
  4. The memory of my system is as follows. I think the memory won't be a problem. It's wired to get this error. total used free shared buff/cache available Mem: 64100 8993 8804 343 46302 54053 Swap: 2047 354 1693
  5. Here is the log of statistical info: statistical_info.txt
  6. I also tried to run the unit test in the pypardiso package. It gave me this error image

PS: I used to use this library for the same code on an old desktop: Ubuntu 20.04 Intel i5 cores=4, threads=8. It worked. Now I am moving the code to another computer, the simple test example run successfully alone, but failed in my project and in the jupyter notebook.

haasad commented 2 years ago

This looks very similar to what you describe: https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/pardiso-insufficient-memory/m-p/1358741#M32717%3Fprofile.language=en

Can you try downgrading mkl and intel-openmp and see if that changes anything?

pip install mkl==2022.0.1 intel-openmp==2022.0.1
shushu-qin commented 2 years ago

Downgrading mkl and intel-openmp doesn't change anything. I circumvented the problem in an inefficient way: save the matrix in a file and start a new process to read the file and perform calculation...