trouble installing

[dswah]

Hi Yotam, I'm very excited about using PySPQR! I can't seem to find a sparse QR implementation for python... But i'm having trouble using the library.

namely, when i import spqr i get an error with the gcc.

Do you know what is going on?

In [1]: import spqr
=== Wrapper module not compiled; compiling...
generating ./_spqr.c
(already up-to-date)
running build_ext
building '_spqr' extension
gcc -fno-strict-aliasing -I/Users/daniel/miniconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/daniel/miniconda/include/python2.7 -c _spqr.c -o ./_spqr.o
_spqr.c:434:10: fatal error: 'SuiteSparseQR_C.h' file not found
#include <SuiteSparseQR_C.h>

-danny

[yig]

Do you have SuiteSparse installed? I have another user on Linux who had what may be the same issue with include paths. @zfergus2 found that making the following change to lines 11-15 in spqr_gen.py worked:

ffibuilder.set_source( "_spqr",
    """#include <SuiteSparseQR_C.h>""",
    include_dirs = ['/usr/include/suitesparse'],
    libraries=['spqr'])

[dswah]

😄 I have like 2 SuiteSparse installed...

My coworker just pointed me a few minutes ago to the Conda version of SuiteSparse, and that worked! https://github.com/conda-forge/suitesparse-feedstock

I will keep your suggestion in mind though for next time. Now i can import spqr!

[dswah]

Hi Yotam,

I'm wondering if really my installation is still wrong... Have you every seen slow performance due to wrong installation?

Im seeing spqr.qr slower than the dense scipy qr on a sparse matrix. Even SVD is faster:

[yig]

How big is your matrix?

On Feb 1, 2017, at 3:59 PM, daniel servén notifications@github.com wrote:

Hi Yotam,

I'm wondering if really my installation is still wrong... Have you every seen slow performance due to wrong installation?

Im seeing spqr.qr slower than the dense scipy qr on a sparse matrix. Even SVD is faster:

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[dswah]

i was using a 10,000 x 53 matrix. with nnz=99,000. maybe this is too non-sparse?

either way if i use the sample code from the readme i see this:

is this expected?

[yig]

I don't think anything is wrong with your installation. For spqr.qr() I get: 1 loop, best of 3: 351 ms per loop For numpy.linalg.qr() I get: 1 loop, best of 3: 181 ms per loop Those numbers look similar to yours.

It is very easy to end up with a dense Q: http://mathoverflow.net/questions/94198/sparsity-of-qr-decomposition Random matrices are triggering this scenario. If the rank of your matrix is much smaller than the dimensions, then you will see a big win. Or with other special scenarios:

In [1]: import spqr
In [2]: import scipy.sparse
In [3]: import numpy as np
In [4]: M = scipy.sparse.identity( 10000 )
In [5]: %timeit Q, R, E, rank = spqr.qr( M )
100 loops, best of 3: 6.92 ms per loop
In [6]: %timeit Q, R = np.linalg.qr( M.todense() )
1 loop, best of 3: 10.6 s per loop

The sparse version used ~200 MB of RAM, while the dense version used ~ 4 GB.

I get similar performance with this sum of a large identity matrix and some randomness in the upper-left:

In [1]: import spqr
In [2]: import scipy.sparse
In [3]: import numpy as np
In [4]: M1 = scipy.sparse.rand( 1000, 1000, density = 0.01 ).tocoo()
In [5]: M2 = scipy.sparse.identity( 10000 ).tocoo()
In [6]: M = scipy.sparse.coo_matrix( ( np.concatenate( ( M1.data, M2.data ) ), ( np.concatenate( ( M1.row, M2.row ) ), np.concatenate( ( M1.col, M2.col ) ) ) ) ) 
In [7]: %timeit Q, R, E, rank = spqr.qr( M )
1 loop, best of 3: 267 ms per loop
In [8]: %timeit Q, R = np.linalg.qr( M.todense() )
1 loop, best of 3: 11.6 s per loop

[yig]

I committed the include_dirs change since it makes it work for linux and is harmless on mac os.

[dswah]

:) ok! awesome.

also, cheers for the sweet python package.

[yig]

Thanks! I'm glad you like it.