search

rezakj / iCellR

Single (i) Cell R package (iCellR) is an interactive R package to work with high-throughput single cell sequencing technologies (i.e scRNA-seq, scVDJ-seq, scATAC-seq, CITE-Seq and Spatial Transcriptomics (ST)).
120 stars 19 forks source link

Memory management problem #12

Closed ehcw111 closed 4 years ago

ehcw111 commented 4 years ago

Hello,

I am trying to test out iCellR, however, I can't seem to get through the data aggregation step. As soon as I load one 10x dataset, there will be a >80% system memory usage and the aggregation step will fail as it cannot allocate a vector of 274kb. This is on a system with 8gb ram with nothing else running. Using a system with 16gb ram seems to work fine, but memory usage was still >80%. I am wondering if there is a way to use iCellR with less than 16gb ram?

Thanks!

rezakj commented 4 years ago

Hi thank you for posting this, a few others were asking about the memory issues with merging the files.

This step is just merging the tow/more data frames into one data frame and is using the basic R merge function. This function in R is known to be very RAM/memory intensive and that's why this happens. But you can use any other method to put all your files together, like using bash join command or python. But soon in a new version of iCellR I will probably use something other than the merge function to prevent this from happening for the users who run their data locally.

Reza

rezakj commented 4 years ago

This is fixed. Use version 1.5.0.

Reza