search

aertslab / scenicplus

SCENIC+ is a python package to build gene regulatory networks (GRNs) using combined or separate single-cell gene expression (scRNA-seq) and single-cell chromatin accessibility (scATAC-seq) data.
Other
178 stars 28 forks source link

how to exporting when file was too big #189

Closed rdf1993 closed 1 year ago

rdf1993 commented 1 year ago

2023-07-31 03:11:21,133 SCENIC+_wrapper INFO Exporting to loom file 2023-07-31 03:11:21,134 SCENIC+ INFO Formatting data 2023-07-31 03:11:27,755 SCENIC+ INFO Creating minimal loom 2023-07-31 03:11:49,516 SCENIC+ INFO Adding annotations 2023-07-31 03:11:52,989 SCENIC+ INFO Adding clusterings 2023-07-31 03:11:53,112 SCENIC+ INFO Adding markers 2023-07-31 03:11:53,649 SCENIC+ INFO Exporting 2023-07-31 03:12:00,465 SCENIC+ INFO Formatting data [1] 4093018 killed python3

SeppeDeWinter commented 1 year ago

@rdf1993

Could you please provide a bit more context?

From the fact that python is killed I guess you ran out of memory, how much do you have available?

Best,

Seppe

rdf1993 commented 1 year ago

thank you for your response. The total memory of server is 256G. I've noticed that I'm exceeding memory limits during the process of exporting region LOOM files. Is it possible to split the LOOM file output and then merge them?

SeppeDeWinter commented 1 year ago

Hi @rdf1993

I would suggest skipping exporting loom files in the wrapper script, i.e. set export_to_loom_file to False. Afterwards you can still export a loom file if you wish (this one you can make smaller before exporting).

Best,

Seppe

rdf1993 commented 1 year ago

Thank you very much. I will try to skip exporting loom files first

SeppeDeWinter commented 1 year ago

Great!

I'm closing this issue for now, feel free to reopen if you still face issues.

Best,

Seppe