neuropsychology / NeuroKit

NeuroKit2: The Python Toolbox for Neurophysiological Signal Processing
https://neuropsychology.github.io/NeuroKit
MIT License
1.57k stars 419 forks source link

Offering help to investigate claim that cvxEDA is broken #781

Closed lciti closed 1 year ago

lciti commented 1 year ago

I am the developer of the cvxEDA code used in NeuroKit.

I would like to offer my help to investigate what sort of issues cvxEDA is giving and to address them, with reference to the comment below in the code: https://github.com/neuropsychology/NeuroKit/blob/5cd089e9d82e6f7fb98d2028e69840aee08f8ab3/neurokit2/eda/eda_phasic.py#L17

There are two related issues in the cvxEDA repository (https://github.com/lciti/cvxEDA/issues/9 and https://github.com/lciti/cvxEDA/issues/10). The error reported there was raised by the cvxopt library with message "ValueError: Rank(A) < p or Rank([P; A; G]) < n". I have downloaded data that was kindly provided by the user who reported the error and installed the same version of the libraries but I could not reproduce the error under any conditions.

Because cvxEDA depends on cvxopt, which in turn depends on cholmod, which in turn depends on a BLAS/LAPACK implementation, it's very hard for me to reproduce the error and find a fix, unless I have the exact same data and libraries. I would be grateful if any person who receives this error could please export the exact list of packages with conda env export > env.yaml and attach it in reply to this issue, together with the data and platform. I am only interested in "reasonable" data and values of the parameters (e.g. the data cannot be just a handful of samples long, the sampling frequency cannot be 0.2 Hz or 500 MHz, because in these cases an error is reasonable).

There is the possibility that the error only happened with a specific platform and version of some of the libraries and that the underlying error in those libraries has then been fixed. In this case, I propose we monitor how cvxEDA is doing now and if it is not giving that error any more, we assume the problem has disappeared.

Thanks!

welcome[bot] commented 1 year ago

Hi 👋 Thanks for reaching out and opening your first issue here! We'll try to come back to you as soon as possible. ❤️ kenobi

DominiqueMakowski commented 1 year ago

Hi @lciti thanks for chiming in here. I was myself a big user of cvxEDA during my PhD, so I am really grateful and happy to see that it's being worked on ☺️

I have updated the latest cvxopt and here is a reproducible code snippet that errors for me:

import neurokit2 as nk

# Simulate EDA signal
eda_signal = nk.eda_simulate(duration=30, scr_number=5, drift=0.1)

nk.signal_plot(eda_signal)

image

cvxEDA = nk.eda_phasic(nk.standardize(eda_signal), method='cvxeda')
---------------------------------------------------------------------------
ArithmeticError                           Traceback (most recent call last)
File c:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\cvxopt\misc.py:1432, in kkt_chol2.<locals>.factor(W, H, Df)
   [1431](file:///c%3A/Users/user/AppData/Local/Programs/Python/Python310/lib/site-packages/cvxopt/misc.py?line=1430)         F['Sf'] = cholmod.symbolic(F['S'])
-> [1432](file:///c%3A/Users/user/AppData/Local/Programs/Python/Python310/lib/site-packages/cvxopt/misc.py?line=1431)         cholmod.numeric(F['S'], F['Sf'])
   [1433](file:///c%3A/Users/user/AppData/Local/Programs/Python/Python310/lib/site-packages/cvxopt/misc.py?line=1432) except ArithmeticError:

ArithmeticError: 5

During handling of the above exception, another exception occurred:

ArithmeticError                           Traceback (most recent call last)
File c:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\cvxopt\coneprog.py:2065, in coneqp(P, q, G, h, dims, A, b, initvals, kktsolver, xnewcopy, xdot, xaxpy, xscal, ynewcopy, ydot, yaxpy, yscal, **kwargs)
   [2064](file:///c%3A/Users/user/AppData/Local/Programs/Python/Python310/lib/site-packages/cvxopt/coneprog.py?line=2063) for rti in W['rti']: rti[::rti.size[0]+1 ] = 1.0
-> [2065](file:///c%3A/Users/user/AppData/Local/Programs/Python/Python310/lib/site-packages/cvxopt/coneprog.py?line=2064) try: f = kktsolver(W)
   [2066](file:///c%3A/Users/user/AppData/Local/Programs/Python/Python310/lib/site-packages/cvxopt/coneprog.py?line=2065) except ArithmeticError:

File c:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\cvxopt\coneprog.py:1981, in coneqp.<locals>.kktsolver(W)
   [1980](file:///c%3A/Users/user/AppData/Local/Programs/Python/Python310/lib/site-packages/cvxopt/coneprog.py?line=1979) def kktsolver(W):
-> [1981](file:///c%3A/Users/user/AppData/Local/Programs/Python/Python310/lib/site-packages/cvxopt/coneprog.py?line=1980)     return factor(W, P)

File c:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\cvxopt\misc.py:1447, in kkt_chol2.<locals>.factor(W, H, Df)
   [1446](file:///c%3A/Users/user/AppData/Local/Programs/Python/Python310/lib/site-packages/cvxopt/misc.py?line=1445)         F['Sf'] = cholmod.symbolic(F['S'])
-> [1447](file:///c%3A/Users/user/AppData/Local/Programs/Python/Python310/lib/site-packages/cvxopt/misc.py?line=1446)         cholmod.numeric(F['S'], F['Sf'])
   [1448](file:///c%3A/Users/user/AppData/Local/Programs/Python/Python310/lib/site-packages/cvxopt/misc.py?line=1447) F['firstcall'] = False
...
   [2073](file:///c%3A/Users/user/AppData/Local/Programs/Python/Python310/lib/site-packages/cvxopt/coneprog.py?line=2072) #     [ A   0   0  ] * [ y ] = [  b ].
   [2074](file:///c%3A/Users/user/AppData/Local/Programs/Python/Python310/lib/site-packages/cvxopt/coneprog.py?line=2073) #     [ G   0  -I  ]   [ z ]   [  h ]
   [2076](file:///c%3A/Users/user/AppData/Local/Programs/Python/Python310/lib/site-packages/cvxopt/coneprog.py?line=2075) xcopy(q, x)

ValueError: Rank(A) < p or Rank([P; A; G]) < n

Do you have the same?

lciti commented 1 year ago

Nope :-( Works like a charm. The data are generated with some random noise, so I've run it 30 times but still no realisation gave any errors.

It may be deep into some library. I've created a new environment with Python 10 (same as you as far as I can see) and the most up to date versions of the libraries. I work under Linux and maybe the same version of cvxopt (1.3.0) might have been compiled with different versions of the libraries in the two architectures or even simply with different building tools/settings.

How did you install your version of cvxopt? If you use anaconda/miniconda, can you please run conda env export > env.yaml. Alternatively, how did you install cvxopt? Do you compile it on your machine or use a wheel? If the latter, what's its exact name?

In the meanwhile I will try it on a Windows machine.

lciti commented 1 year ago

Tried with Windows, conda create -n testEDApy310b -c conda-forge --override-channels python=3.10 neurokit2 cvxopt, and it works there too (I've used random_seed=1234, does it give error to you?).

After some digging, I've seen that the exact version being used is cvxopt-1.3.0-py310h6b0d2d9_4. Could you please check which one is used in your system? Alternatively, could you download the version from the link above, then open the compressed file and copy the folder Lib/site-packages/cvxopt into the folder from where you run your test (please use a seed so we can use the same data). Now, when you run the test, the local version of cvxopt is used (you can try adding a print('Hi!') inside the function cvxopt/coneprog.py/coneqp).

Thanks!

lciti commented 1 year ago

Could anyone reporting errors please run this script: versions.py.zip

DominiqueMakowski commented 1 year ago

Here is the data for the system where it errors:

platform Windows-10-10.0.22621-SP0
python 3.9.7 (tags/v3.9.7:1016ef3, Aug 30 2021, 20:19:38) [MSC v.1929 64 bit (AMD64)]
cvxopt 1.3.0
- OS: Windows (WindowsPE 64bit) 
- Python: 3.9.7 
- NeuroKit2: 0.2.4 
- NumPy: 1.20.0 
- Pandas: 1.4.3 
- SciPy: 1.10.0 
- sklearn: 1.0.2 

36ed6bc07d20b30838ee6850be99ffe477b9c26bf750f99c39a9308a7b5034e9 amd.cp39-win_amd64.pyd
0a111c524cb942631e2fa5eca64dd04b4e3b9b617c4a02399a82d3acf39202e6 base.cp39-win_amd64.pyd
32fa1b0a376db231e5dcc40b08bc253b77487e9aae0cf9df109abdf14a554a62 blas.cp39-win_amd64.pyd
b5d71d53849d2d719cacc6688b4279932e07445ddb1a1ada030a128cc2c9de73 cholmod.cp39-win_amd64.pyd
9c866c35a6a27346a73c5e6bd3a0cc1ec0c146116f24f0d9c48a80e5ad9a03e2 coneprog.py
ed565bcfd7410a3d8fe45d7f1c8ddf2564850d3c137c3b9671df2e2170e88b85 cvxprog.py
1076760af5ab62830410167396708dfff3a76a0c5d003d7e2baa6cca6cc6f7e2 glpk.cp39-win_amd64.pyd
f76cca5ba65ff76b85a55aba60372a293f855ab6737cf91bedd515e47547d04b info.py
5d6a01a36a5a50b6d56924058190114fdefaff53d607b4fa80a213abead99c35 lapack.cp39-win_amd64.pyd
190b5a9c8d9723fe958ad33916bd7346d96fab3c5ea90832bb02d854f620fcff LICENSE_OpenBLAS-0.3.19
020fec6af9637323d17c3c6fd7843066db1c748594853455a4bf677f83f2d702 misc.py
42efe5be23bf6ea2bd2943cd3e8a187f263982a8b173fe65d24183261ae3e57a misc_solvers.cp39-win_amd64.pyd
f1784989e691ee9125a2176f67e91fc6a895a7f34c093b61a0dcf5873303411e modeling.py
62eb521199d85a6af6b4e309ded8bc1b8aa0bc504448a0a6427c021cf4ee9233 msk.py
1b9cb6de90a0ad0551a0577b1422fbf2063c64764a6b1cc4af43c9f2e26535b0 printing.py
8f10f8058ec2295a6e0591562ea96551f933f69745cdd70b965e3396a0847371 solvers.py
8db975786895cac3873bf4d5674743bf91afd89613d2287a18dd9ef813c3d41d umfpack.cp39-win_amd64.pyd
2ddd0e769bbcf5ca23b45cbdf15b0bac4a627dfbf9571349aaa42298d845b9b6 _openblas_bootstrap.py
f108dc5bc03338a98936fa515ed555bccfc85f05be427360314da3334f0878dc _version.py
c1255ecbd38071b99ac51ab1fb5275ed3426ca7d413e5964190a39164dcb5820 __init__.py
DominiqueMakowski commented 1 year ago

And this is on another machine:

platform Windows-10-10.0.19042-SP0
python 3.10.3 (tags/v3.10.3:a342a49, Mar 16 2022, 13:07:40) [MSC v.1929 64 bit (AMD64)]
cvxopt 1.3.0
8f005e3e81c84f13e5270779b1c1feda2d2b63dccae5293311a1302c91602011 amd.cp310-win_amd64.pyd
02e3c0212247d76ec63153c359986b6bf34802db88b57f2dae5f47ef0919a16d base.cp310-win_amd64.pyd
116467bcd3bc9d48c3c2abbf89ea3e2bbb25c8078a7e5b95816ea9e1bc029b94 blas.cp310-win_amd64.pyd
894489cd941739176d014f6d3f24494f7af7e82ac8474b6240941c9f060490d7 cholmod.cp310-win_amd64.pyd
9c866c35a6a27346a73c5e6bd3a0cc1ec0c146116f24f0d9c48a80e5ad9a03e2 coneprog.py
ed565bcfd7410a3d8fe45d7f1c8ddf2564850d3c137c3b9671df2e2170e88b85 cvxprog.py
a9fb8245829d53831c0454dc0a0efa8f9b76a1496c81c570c72d96ec87d38432 glpk.cp310-win_amd64.pyd
f76cca5ba65ff76b85a55aba60372a293f855ab6737cf91bedd515e47547d04b info.py
312a66c410776ab45b61df60ead4b98a10f9b66b4532a7b8afd3c54f529cc84a lapack.cp310-win_amd64.pyd
190b5a9c8d9723fe958ad33916bd7346d96fab3c5ea90832bb02d854f620fcff LICENSE_OpenBLAS-0.3.19
020fec6af9637323d17c3c6fd7843066db1c748594853455a4bf677f83f2d702 misc.py
f1490dbad7c7cccc80db37f44b70d2c6105316e63e2f0cda0a89a77030b1a667 misc_solvers.cp310-win_amd64.pyd
f1784989e691ee9125a2176f67e91fc6a895a7f34c093b61a0dcf5873303411e modeling.py
62eb521199d85a6af6b4e309ded8bc1b8aa0bc504448a0a6427c021cf4ee9233 msk.py
1b9cb6de90a0ad0551a0577b1422fbf2063c64764a6b1cc4af43c9f2e26535b0 printing.py
8f10f8058ec2295a6e0591562ea96551f933f69745cdd70b965e3396a0847371 solvers.py
88458f84daab78ba0a403ad63071fae2e12389a4a056869622a7e98a266e37f5 umfpack.cp310-win_amd64.pyd
2ddd0e769bbcf5ca23b45cbdf15b0bac4a627dfbf9571349aaa42298d845b9b6 _openblas_bootstrap.py
f108dc5bc03338a98936fa515ed555bccfc85f05be427360314da3334f0878dc _version.py
c1255ecbd38071b99ac51ab1fb5275ed3426ca7d413e5964190a39164dcb5820 __init__.py
lciti commented 1 year ago

Both of them throw the same type of error? What seed have you used for nk.eda_simulate? Does it error for some/most/all seeds?

lciti commented 1 year ago

Let me focus on the Python 3.10 version. Those hashes correspond to the pip version cvxopt-1.3.0-cp310-cp310-win_amd64.whl available here. I have downloaded and I have run it several times and it works just fine. Here is my system:

platform Windows-10-10.0.19044-SP0
python 3.10.9 | packaged by conda-forge | (main, Feb  2 2023, 20:14:58) [MSC v.1929 64 bit (AMD64)]
neurokit2 0.2.3
cvxopt 1.3.0
8f005e3e81c84f13e5270779b1c1feda2d2b63dccae5293311a1302c91602011 amd.cp310-win_amd64.pyd
02e3c0212247d76ec63153c359986b6bf34802db88b57f2dae5f47ef0919a16d base.cp310-win_amd64.pyd
116467bcd3bc9d48c3c2abbf89ea3e2bbb25c8078a7e5b95816ea9e1bc029b94 blas.cp310-win_amd64.pyd
894489cd941739176d014f6d3f24494f7af7e82ac8474b6240941c9f060490d7 cholmod.cp310-win_amd64.pyd
...

I have noted that cholmod.cp310-win_amd64.pyd appears to depend on libopenblas.dll (which is not shipped inside cvxopt) and I am now trying to investigate if/how this may be the problem.

lciti commented 1 year ago

@DominiqueMakowski Could you please try to run this file: portfolio.py.zip It should give a row vector close to 0 (~2e-15 or less) and vector with just [1.].

It comes from issue https://github.com/cvxopt/cvxopt/issues/226 which is similar to the one we are seeing here: some people report an error with cvxopt 1.3.0 (pip version) on Windows and say that downgrading to 1.2.6 fixes the problem. The maintainer cannot reproduce it on their side. It may be that the underlying problem is the same.

... And if you have the chance, please try this one too cvxopt_issue_230.zip which comes from https://github.com/cvxopt/cvxopt/issues/230. It should give:

     pcost       dcost       gap    pres   dres
 0:  8.6380e+02 -1.0551e+04  2e+04  2e-01  3e-15
 1:  2.5019e+02 -1.2611e+03  2e+03  1e-02  2e-15
 2: -5.9149e+01 -4.0743e+02  4e+02  1e-03  1e-15
 3: -1.2597e+02 -2.2293e+02  1e+02  2e-04  9e-16
 4: -1.4107e+02 -1.6582e+02  2e+01  1e-05  9e-16
 5: -1.4472e+02 -1.4957e+02  5e+00  7e-15  1e-15
 6: -1.4551e+02 -1.4638e+02  9e-01  4e-16  8e-16
 7: -1.4567e+02 -1.4572e+02  5e-02  7e-15  9e-16
 8: -1.4568e+02 -1.4568e+02  4e-03  2e-15  1e-15
 9: -1.4568e+02 -1.4568e+02  6e-05  9e-15  9e-16
Optimal solution found.
0.931156431730394

But it is reported to reach the maximum number of iterations (100) without converging. It works for me on all machines I have access to (Linux and Windows) and all versions of cvxopt that I have tried (conda, pip, etc).

DominiqueMakowski commented 1 year ago

The first file got me this:

[[ 1.16573418e-15  1.27675648e-15 -1.41553436e-15 -1.33226763e-15
   3.05311332e-16 -1.11022302e-15 -4.44089210e-16]]
[1.]

And second got me:

     pcost       dcost       gap    pres   dres
 0:  5.8094e+03 -1.4125e+04  3e+04  8e+01  2e+01
 1:  5.4687e+02 -1.0686e+04  2e+04  3e+02  1e+01
 2:  4.8454e+02 -1.0925e+04  1e+04  2e+02  4e+00
 3:  9.8114e+02 -5.5224e+03  7e+03  4e+01  5e+00
 4:  1.2846e+03 -5.8194e+03  7e+03  7e+01  1e+01
 5:  9.9353e+02 -3.5111e+03  2e+03  2e+02  6e+00
 6:  1.0632e+03 -3.9064e+03  2e+03  2e+02  9e+00
 7:  1.4707e+03 -4.2776e+03  2e+03  3e+02  1e+01
 8:  2.6336e+03 -6.0241e+03  3e+03  4e+02  1e+01
 9:  2.9885e+03 -7.6128e+03  4e+03  4e+02  2e+01
10:  2.6895e+03 -6.6977e+03  4e+03  4e+02  9e+00
11:  2.3917e+03 -5.5886e+03  4e+03  4e+02  8e+00
12:  2.2350e+03 -4.7434e+03  4e+03  4e+02  3e+01
13:  1.8243e+03 -4.0638e+03  4e+03  3e+02  7e+01
14:  1.2980e+03 -3.3454e+03  4e+03  3e+02  1e+02
15:  5.7026e+02 -2.6882e+03  4e+03  1e+02  3e+02
16:  4.9397e+02 -2.5331e+03  4e+03  1e+02  5e+02
17:  6.8811e+02 -3.8826e+03  4e+03  9e+01  3e+02
18:  1.2807e+03 -5.5551e+03  3e+03  4e+02  2e+02
19:  1.7245e+03 -9.8109e+03  4e+03  5e+02  5e+02
20:  1.7471e+03 -9.9289e+03  4e+03  5e+02  5e+02
21:  1.7910e+03 -1.0005e+04  4e+03  5e+02  5e+02
22:  1.8942e+03 -1.0130e+04  4e+03  5e+02  5e+02
23:  3.6339e+03 -1.0694e+04  4e+03  7e+02  2e+03
24:  3.6322e+03 -1.0438e+04  4e+03  7e+02  3e+05
25:  3.6330e+03  8.5994e+05  4e+03  7e+02  1e+09
26:  3.6330e+03  9.3321e+06  4e+03  7e+02  1e+10
27:  3.6318e+03  9.8714e+10  2e+06  7e+02  1e+14
28:  3.6349e+03  1.2672e+14  7e+07  7e+02  2e+17
29:  3.6349e+03  9.1235e+14  5e+08  7e+02  1e+18
30:  3.6294e+03  7.1850e+18  4e+12  7e+02  9e+21
31:  3.6418e+03  1.5348e+21  4e+14  7e+02  2e+24
32:  3.6420e+03  7.6169e+21  2e+15  7e+02  1e+25
33:  4.9004e+03  1.3159e+24  3e+17  2e+02  2e+27
34:  4.8334e+03  1.1351e+25  3e+18  2e+02  1e+28
35:  1.3220e+04  1.4760e+25  3e+18  6e+01  2e+28
36:  3.6609e+04  6.7470e+25  2e+19  7e+02  8e+28
37:  3.6512e+04  4.4211e+26  1e+20  7e+02  6e+29
38:  1.5736e+04  4.8660e+26  1e+20  5e+01  6e+29
39:  1.5736e+04  2.6144e+27  6e+20  5e+01  3e+30
40:  3.6910e+04  2.7345e+27  6e+20  7e+02  3e+30
41:  3.4691e+04  2.7575e+27  6e+20  6e+02  3e+30
42:  3.3440e+04  2.7921e+27  6e+20  6e+02  3e+30
43:  3.3440e+04  9.7468e+29  2e+23  6e+02  1e+33
44:  1.6131e+04  9.7525e+29  2e+23  7e+01  1e+33
45:  3.7138e+04  9.7779e+29  2e+23  7e+02  1e+33
46:  3.7138e+04  8.9086e+30  2e+24  7e+02  1e+34
47:  1.7930e+04  8.9129e+30  2e+24  1e+02  1e+34
48:  3.3117e+04  8.9253e+30  2e+24  6e+02  1e+34
49:  3.2686e+04  8.9534e+30  2e+24  6e+02  1e+34
50:  3.2098e+04  9.0286e+30  2e+24  6e+02  1e+34
51:  2.6050e+04  1.2781e+31  3e+24  4e+02  2e+34
52:  2.6050e+04  6.5568e+31  1e+25  4e+02  8e+34
53:  2.3281e+04  6.5688e+31  2e+25  3e+02  8e+34
54:  2.5454e+04  6.6334e+31  2e+25  4e+02  8e+34
55:  2.5099e+04  4.1637e+32  1e+26  4e+02  5e+35
56:  2.5153e+04  4.1637e+32  1e+26  4e+02  5e+35
57:  2.4629e+04  4.1640e+32  1e+26  4e+02  5e+35
58:  2.3424e+04  4.1777e+32  1e+26  3e+02  5e+35
59:  2.4543e+04  4.2302e+32  1e+26  4e+02  5e+35
60:  2.4543e+04  6.5876e+33  2e+27  4e+02  8e+36
61:  2.4913e+04  6.5898e+33  2e+27  4e+02  8e+36
62:  2.4542e+04  6.5990e+33  2e+27  4e+02  8e+36
63:  2.5362e+04  6.6597e+33  2e+27  4e+02  8e+36
64:  2.4812e+04  6.6677e+33  2e+27  4e+02  8e+36
65:  2.4812e+04  3.3458e+34  8e+27  4e+02  4e+37
66:  2.4715e+04  3.3486e+34  8e+27  4e+02  4e+37
67:  2.4976e+04  3.3707e+34  8e+27  4e+02  4e+37
68:  2.4976e+04  2.1204e+35  5e+28  4e+02  3e+38
69:  2.4956e+04  2.1205e+35  5e+28  4e+02  3e+38
70:  2.5556e+04  2.1207e+35  5e+28  4e+02  3e+38
71:  2.5565e+04  2.1210e+35  5e+28  4e+02  3e+38
72:  2.4881e+04  2.1212e+35  5e+28  4e+02  3e+38
73:  2.4812e+04  2.1213e+35  5e+28  4e+02  3e+38
74:  2.5264e+04  2.1214e+35  5e+28  4e+02  3e+38
75:  2.5322e+04  2.1217e+35  5e+28  4e+02  3e+38
76:  2.4691e+04  2.1218e+35  5e+28  4e+02  3e+38
77:  2.5444e+04  2.1234e+35  5e+28  4e+02  3e+38
78:  2.4633e+04  2.1353e+35  5e+28  4e+02  3e+38
79:  2.4229e+04  1.1165e+36  3e+29  4e+02  1e+39
80:  2.5048e+04  1.1171e+36  3e+29  4e+02  1e+39
81:  2.5048e+04  5.6716e+36  1e+30  4e+02  7e+39
82:  2.5103e+04  5.6717e+36  1e+30  4e+02  7e+39
83:  2.4426e+04  5.6721e+36  1e+30  4e+02  7e+39
84:  2.4373e+04  5.6730e+36  1e+30  4e+02  7e+39
85:  2.5102e+04  5.7388e+36  1e+30  4e+02  7e+39
86:  2.4837e+04  2.3986e+37  5e+30  4e+02  3e+40
87:  2.4317e+04  2.4091e+37  6e+30  4e+02  3e+40
88:  2.4317e+04  1.9358e+38  4e+31  4e+02  2e+41
89:  2.5183e+04  1.9360e+38  4e+31  4e+02  2e+41
90:  2.4565e+04  1.9392e+38  4e+31  4e+02  2e+41
Terminated (singular KKT matrix).
4.9852842902256524
lciti commented 1 year ago

Thank you very much! Can you confirm if this is the "python 3.9.7" or the "python 3.10.3" versions above of which you have provided hashes?

DominiqueMakowski commented 1 year ago

3.9.7

lciti commented 1 year ago

Hi @DominiqueMakowski When you have a chance, could you please try this code cvxopt_issue_230_2.zip, e.g. with python cvxopt_issue_230_2.py > log.txt and send me the log file? Thanks!

lciti commented 1 year ago

@DominiqueMakowski, we have made some good progress in https://github.com/cvxopt/cvxopt/issues/230. I am confident we got to the bottom of that issue (a problem with OpenBLAS v0.3.19 on SkyLakeX architectures), which I believe is also the cause of this one. The maintainer of cvxopt has released new pip wheels that link a newer version of OpenBLAS and this should fix the problem. If you have the chance, could you please give it a try and see if installing the new wheels from Test PyPI makes https://github.com/cvxopt/cvxopt/issues/230 and https://github.com/neuropsychology/NeuroKit/issues/781 disappear? Many thanks!

DominiqueMakowski commented 1 year ago

I just checked, it seems like it works 🥳

DominiqueMakowski commented 1 year ago

Thanks a ton Luca for digging to the bottom of this. I've been following the cvxopt thread, super impressive! ☺️

BTW, I was recently checking the advances in EDA decomposition, and I was wondering - out of curiosity -, now that several years have passed since the original publication of cvxEDA, if you had any new intuitions for potential improvements to this algo, or thoughts as to potentially better approaches to explore?

lciti commented 1 year ago

Some years ago I have been working on a version of cvxEDA with a sparse prior but then never found the time to finish it. I meant to write you and talk about it and possibly think of a project together. I'll contact you on your Sussex Uni email.