Open j-i-l opened 3 months ago
https://github.com/alexbovet/flow_stability/pull/35#issuecomment-2306467560
We might want to add the possibility to install flowstab
in "large" data mode where we consistently use long long
for all index arrays.
Yes, that could be a good option.
I remember there was an issue with the connected_components function from scipy.csgraph which cannot use int64 (here. Something to keep in mind.
Also, I think this is a discussion they have at scipy, and other projects.
In C++ we could simply use template functions which would allow us not having to specify the type of indices
and indptr
.
I'm unsure if/how this can be done in cython...
Actually, scipy
as nice examples of this approach in their sparsetools
:
That might be a good approach for us: We would only need to set the type in the numpy representation, i.e. np.int32
or np.int64
and the rest would follow... well that would be the idea.
When passing the arguments of a
scipy.sparse.csr_matrix
tosparse_stoch_from_full_csr
we run into aValueError
:_See e.g. here_
We can address this by explicitly converting the indices to
np.int64
when passing them as arguments, or we might change datat type toint
instead oflong long
insparse_stochfrom_full_csr
(and others).Note that for some functions in __cython_sparsestoch.pyx already declare
int[:]
as data type when indices are passed as arguments.https://github.com/alexbovet/flow_stability/actions/runs/10513029536/job/29127746857#step:5:116
It also seems not completely consistent in the defs:
E.g. for
indptr
some functions requireint
:https://github.com/alexbovet/flow_stability/blob/8648a8bd715e3b086ed13cb7453d41754c176398/src/flowstab/_cython_sparse_stoch.pyx#L859-L863
some
long long
: https://github.com/alexbovet/flow_stability/blob/8648a8bd715e3b086ed13cb7453d41754c176398/src/flowstab/_cython_sparse_stoch.pyx#L613-L618I'd put everything
indices
andindptr
tolong
and declare withnp.int64
on the python site. @alexbovet is that OK for you or is there actually a use-case forlong long
?_Originally posted by @j-i-l in https://github.com/alexbovet/flow_stability/issues/35#issuecomment-2305337823_