search

dnanexus-archive / viral-ngs

viral-ngs
6 stars 6 forks source link

scale back deplete instance size #57

Closed dpark01 closed 7 years ago

dpark01 commented 7 years ago

Looking at the logs in our deplete executions, it looks to be horribly utilizing the resources we're giving it. It's currently getting a pricey mem3_x16, but it never seems to use more than 12GB RAM, 2 cores at best, maybe about 80GB disk.

Until we figure out how to parallelize better on our end, can you change this instance to a mem1_ssd1_x8 or mem2_ssd1_x4 or whatever seems economical?

mlin commented 7 years ago

OK; FWIW I think the actual blastn runs parallelize nicely; it's RevertSam+FilterSamReads+SamToFastq that spin 1-2 threads for hours on end (on HiSeq runs)