SKA run out of memory on 32 GB VM

simonrharris / SKA

Split Kmer Analysis

MIT License

62 stars 1 forks source link

SKA run out of memory on 32 GB VM #26

Open foamy1881 opened 4 years ago

foamy1881 commented 4 years ago

Please help! I'm running out of memory on 32GB VM when trying to merge more than 1000 isolates of Mtb. It max out ram and whole VM frozen after reading file 389 everytime. Can anyone shed some light on it??

tseemann commented 4 years ago

If it is running out of RAM, there is probably not much you can do except get more RAM, which is not possible. 32 GB is 32000 MB. You have 1000 Mtb genomes which are 4 MB as DNA alone, let alone all the overhead for storing the data structures used in the SKA analysis. An overhead factor of 10x is not unreasonable. So 1000 genomes x 4 MB x 10 overhead = 40 GB RAM.