search

Xinglab / espresso

Other
48 stars 4 forks source link

System sort commands run out of memory #24

Closed chrisamiller closed 8 months ago

chrisamiller commented 1 year ago

The system out calls to sort commands can run out of memory when working with large datasets and cause the whole ESPRESSO_C process to die. Unix sort contains a built in -S option that allows you set the buffer size, and it seems like that it could be set to some reasonable value to help prevent this error.

https://github.com/Xinglab/espresso/blob/091f65dd3291c309a6f8840ed2e51cae1b75cd10/src/ESPRESSO_C.pl#LL160C70-L160C70

sagnikbanerjee15 commented 1 year ago

Hi @chrisamiller,

I think I have the same problem. The ESPRESSO_C step fails without any error. Did you experience something similar?

Thanks.

EricKutschera commented 8 months ago

--sort_buffer_size is added in v1.4.0 with 2G default