search

fraterenz / ecdna-evo

Evolutionary models of extrachromosomal DNA (ecDNA).
0 stars 0 forks source link

Reduce memory usage #17

Closed fraterenz closed 2 years ago

fraterenz commented 2 years ago

Maybe the iter API of rayon is such that it waits until all tasks have been performed instead of dropping them along the way? Required for the cluster #6. Right now, we have for 20 cores more than 640G of memory...

fraterenz commented 2 years ago

It seems that the issue is when we clone the ecDNA distribution here https://github.com/fraterenz/ecdna-evo/blob/7e686864985ab6d3dd367cbcce2e735bbd7593ba/ecdna-evo/src/timepoints.rs#L139

but not sure. However, if we approximate the memory required for simulating 10^9 cells (i.e. considering only the ecDNA distribution), we will get:

memory per core = 10^9 16 bit = 10^9 16 bit 0.125 byte/bit = 2G memory in total = number of cores memory per core + patient data = number of cores * 2G + 2G

fraterenz commented 2 years ago

Fixed by 3159954864064905ad43ebd7df9676f079aa1eb9