Failure to write cell-by-gene matrix as HDF5

hubmapconsortium / sc-atac-seq-pipeline

ATAC-seq pipelines for HuBMAP

MIT License

2 stars 4 forks source link

Failure to write cell-by-gene matrix as HDF5 #35

Closed mruffalo closed 3 years ago

mruffalo commented 3 years ago

As per @jswelling:

Writing cell by gene data to cell_by_gene.hdf5
[1] TRUE
Error in asMethod(object) :
  Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 102
Calls: h5write -> t -> as.matrix -> as.matrix -> as -> asMethod
Execution halted

Fix by writing as MTX, including this as a sparse matrix in the H5AD output, and not exposing the MTX file as a final pipeline output.

mruffalo commented 3 years ago

Fixed in https://github.com/hubmapconsortium/sc-atac-seq-pipeline/commit/039d5c7d17be9d47c020226aaa17c791eeb71faf (which I rebased a few times and then re-committed from scratch), and included in the v1.4 release.