cellxgene_census.experimental.util._csr_iter.X_sparse_iter has been found to be of great utility for both internal and external cases. Since it has not been advertised and it is experimental, users have mentioned the need to gain the functionality that this method has.
Context
The recurrent issue is the following:
I need a memory efficient way to access sparse matrices for downstream analysis. I often require:
An iterator that provides csr chunks
Full rows per chunk, i.e. no disjoint chunks.
Re-indexed positional indices, i.e. no soma_joinids
Potential solutions
Promote this method as stable in cellxgene_census
Find a path to move functionality down to tiledbsoma. Probably requires more design work to ensure we have the best way to integrate with tieldbsoma
Description
cellxgene_census.experimental.util._csr_iter.X_sparse_iter
has been found to be of great utility for both internal and external cases. Since it has not been advertised and it is experimental, users have mentioned the need to gain the functionality that this method has.Context
The recurrent issue is the following:
soma_joinids
Potential solutions
cellxgene_census
tiledbsoma
. Probably requires more design work to ensure we have the best way to integrate withtieldbsoma