Open linmuchuiyang opened 2 years ago
@linmuchuiyang thanks for opening an issue on this. Is there any chance that dataset is publicly available? If not, have you been able to reproduce this behavior on a dataset that is publicly available, or with generated data?
This behavior does seem pretty weird, but it's not the first time I've seen strange behavior like this. Does it also fail if you select a few entries closer to the beginning of the matrix? Something like sparse_gpu_array[:10]
?
However, if I set the interval with [1000:1360000], it will not feedback with any error.
When it doesn’t feedback with any error, are you saying it succeeds or that it seems to fail silently without any error?
In the meantime, I’ll try loading up a couple of the datasets we use in the examples and see if I can reproduce the behavior.
@cjnolet , I've already passed the original datafile to NVIDIA SA. He said he had sent you the email with the data. Is there any feedback, currently.
I encounter this error too, when matrix more than one million cells and 20k genes. I think this error is produce by vstack
when chunks merge and there are produced negative indptr in CSR/CSC matrix, maybe the matrix is over limitation of CSR matrix or CSC matrix (Should < 2^32 elements in matrix?). But I don't have any idea how to fix it.
When I use .get() for the csr_matrix(sparse_gpu_array), an error shown as follows:
However, if I set the interval with [1000:1360000], it will not feedback with any error. The original h5ad file is about 20GiB. And the shape for sparse_gpu_array is (1462703, 27610). Why it will show error for this special interval?