dnanexus-rnd / GLnexus

Scalable gVCF merging and joint variant calling for population sequencing projects
Apache License 2.0
145 stars 38 forks source link

Optimizing memory usage #308

Open georgiiprovisor opened 7 months ago

georgiiprovisor commented 7 months ago

Is it possible to save some intermediate results to reduce memory usage? I'm using GLnexus for a big project (around 5k WES on test, expecting around 30-35k in total). We divided genome in 100 +/- equal pieces and run each piece separately from joint step. So we have 5k full gVCFs and 100 multi-sample VCF. And temporary files for each genome part require a LOT OF storage. Is it possible to optimize these storage usage?