search

walaj / svaba

Structural variation and indel detection by local assembly
GNU General Public License v3.0
235 stars 45 forks source link

Required memory for writing extracted reads #94

Open jykr opened 3 years ago

jykr commented 3 years ago

Hi, I'm trying to extract all discordant reads while running svaba using the --write-extracted-reads option but it failed due to memory limit (30G). How big of memory would I need to have? Currently, I have 162G of extracted reads file written until the failure.

walaj commented 3 years ago

I think what’s happening is that it’s getting a memory load when reading back in the huge bps.txt file to convert to VCF. Can you confirm that it completed the run and produced a full bps.txt.gz file?

On Dec 3, 2020, at 5:28 AM, sehibb notifications@github.com wrote:

 Hi, I'm trying to extract all discordant reads while running svaba using the --write-extracted-reads option but it failed due to memory limit (30G). How big of memory would I need to have? Currently, I have 162G of extracted reads file written until the failure.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

jykr commented 3 years ago

Thanks for extremely fast reply.

Sorry - I lost the result while trying to produce the vcfs by using svaba refilter without specifying -o option. The extracted reads bam is truncated so isn't complete yet. I'll try to work around by turning off --write-extracted-reads option to see if I can produce result. Thanks!

jykr commented 3 years ago

I've tried running without --write-extracted-reads option which still caused the same problem under 30G memory limit. bps.txt file wasn't complete but still is 40Gb.

walaj commented 3 years ago

This is very odd, svaba should be much more judicious with memory. Is this with multi-thread or single thread? Any other tips you could give (e.g. the run command) so I could sort out what may be going on?