search

colomemaria / epiScanpy

Episcanpy: Epigenomics Single Cell Analysis in Python
BSD 3-Clause "New" or "Revised" License
139 stars 33 forks source link

drastically reduce bld_mtx_fly() peak memory footprint #59

Closed le-ander closed 4 years ago

le-ander commented 4 years ago

While bld_mtx_fly() is now running fast enough to create even large count matrices, it would have required 2.5 TB system memory to construct my feature matrix for 10kb windows and hence crashed.

I changed the function to reduce the peak memory consumption significantly.

le-ander commented 4 years ago

Have you had a chance to have a look at this? @DaneseAnna Works fine for my data.