I'm running out of memory despite using a bed file. I'm working with a large genome (zebrafish) so that's not a big surprise (although I'm using up to 256 GB of RAM:
Picked up _JAVA_OPTIONS: -Xms32g -Xmx256g
Yet, I get this error:
Critical exception occurs on region: 2:1-59640629, program will be stopped.
Critical exception occurs on region: 3:1-62628489, program will be stopped.
java.util.concurrent.CompletionException: java.lang.OutOfMemoryError: Java heap space
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314)
at java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:683)
at java.base/java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:658)
at java.base/java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:2094)
at com.astrazeneca.vardict.modes.AbstractMode.pipeline(AbstractMode.java:49)
at com.astrazeneca.vardict.modes.SomaticMode.processBothBamsInPipeline(SomaticMode.java:111)
at com.astrazeneca.vardict.modes.SomaticMode.access$000(SomaticMode.java:31)
at com.astrazeneca.vardict.modes.SomaticMode$SomdictWorker.call(SomaticMode.java:96)
at com.astrazeneca.vardict.modes.SomaticMode$SomdictWorker.call(SomaticMode.java:79)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.OutOfMemoryError: Java heap space
at htsjdk.samtools.SAMUtils.getAlignmentBlocks(SAMUtils.java:742)
at htsjdk.samtools.SAMRecord.getAlignmentBlocks(SAMRecord.java:1789)
at htsjdk.samtools.SAMRecord.validateCigar(SAMRecord.java:1807)
at htsjdk.samtools.BAMRecord.getCigar(BAMRecord.java:284)
at htsjdk.samtools.SAMRecord.isValid(SAMRecord.java:2103)
at htsjdk.samtools.BAMFileReader$BAMFileIterator.advance(BAMFileReader.java:848)
at htsjdk.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:834)
at htsjdk.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:802)
at htsjdk.samtools.BAMFileReader$BAMQueryFilteringIterator.advance(BAMFileReader.java:1079)
at htsjdk.samtools.BAMFileReader$BAMQueryFilteringIterator.(BAMFileReader.java:1048)
at htsjdk.samtools.BAMFileReader.createIndexIterator(BAMFileReader.java:959)
at htsjdk.samtools.BAMFileReader.query(BAMFileReader.java:612)
at htsjdk.samtools.SamReader$PrimitiveSamReaderToSamReaderAdapter.query(SamReader.java:533)
at htsjdk.samtools.SamReader$PrimitiveSamReaderToSamReaderAdapter.query(SamReader.java:538)
at htsjdk.samtools.SamReader$PrimitiveSamReaderToSamReaderAdapter.queryOverlapping(SamReader.java:400)
at com.astrazeneca.vardict.data.SamView.(SamView.java:21)
at com.astrazeneca.vardict.modules.VariationRealigner.noPassingReads(VariationRealigner.java:2060)
at com.astrazeneca.vardict.modules.VariationRealigner.realigndel(VariationRealigner.java:617)
at com.astrazeneca.vardict.modules.VariationRealigner.realignIndels(VariationRealigner.java:387)
at com.astrazeneca.vardict.modules.VariationRealigner.process(VariationRealigner.java:88)
at com.astrazeneca.vardict.modes.AbstractMode$$Lambda$43/0x00007ed9acccf458.apply(Unknown Source)
at java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:680)
at java.base/java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:658)
at java.base/java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:2094)
at com.astrazeneca.vardict.modes.AbstractMode.pipeline(AbstractMode.java:49)
at com.astrazeneca.vardict.modes.SomaticMode.processBothBamsInPipeline(SomaticMode.java:111)
at com.astrazeneca.vardict.modes.SomaticMode.access$000(SomaticMode.java:31)
at com.astrazeneca.vardict.modes.SomaticMode$SomdictWorker.call(SomaticMode.java:96)
at com.astrazeneca.vardict.modes.SomaticMode$SomdictWorker.call(SomaticMode.java:79)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
It is a bit puzzling to me having a "critical exception occurs on region: 2:1-59640629". In fact, this region was split in 10 kb overlapping windows.
Dear developers,
I'm running out of memory despite using a bed file. I'm working with a large genome (zebrafish) so that's not a big surprise (although I'm using up to 256 GB of RAM: Picked up _JAVA_OPTIONS: -Xms32g -Xmx256g
Yet, I get this error: Critical exception occurs on region: 2:1-59640629, program will be stopped. Critical exception occurs on region: 3:1-62628489, program will be stopped. java.util.concurrent.CompletionException: java.lang.OutOfMemoryError: Java heap space at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314) at java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:683) at java.base/java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:658) at java.base/java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:2094) at com.astrazeneca.vardict.modes.AbstractMode.pipeline(AbstractMode.java:49) at com.astrazeneca.vardict.modes.SomaticMode.processBothBamsInPipeline(SomaticMode.java:111) at com.astrazeneca.vardict.modes.SomaticMode.access$000(SomaticMode.java:31) at com.astrazeneca.vardict.modes.SomaticMode$SomdictWorker.call(SomaticMode.java:96) at com.astrazeneca.vardict.modes.SomaticMode$SomdictWorker.call(SomaticMode.java:79) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.OutOfMemoryError: Java heap space at htsjdk.samtools.SAMUtils.getAlignmentBlocks(SAMUtils.java:742) at htsjdk.samtools.SAMRecord.getAlignmentBlocks(SAMRecord.java:1789) at htsjdk.samtools.SAMRecord.validateCigar(SAMRecord.java:1807) at htsjdk.samtools.BAMRecord.getCigar(BAMRecord.java:284) at htsjdk.samtools.SAMRecord.isValid(SAMRecord.java:2103) at htsjdk.samtools.BAMFileReader$BAMFileIterator.advance(BAMFileReader.java:848) at htsjdk.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:834) at htsjdk.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:802) at htsjdk.samtools.BAMFileReader$BAMQueryFilteringIterator.advance(BAMFileReader.java:1079) at htsjdk.samtools.BAMFileReader$BAMQueryFilteringIterator.(BAMFileReader.java:1048)
at htsjdk.samtools.BAMFileReader.createIndexIterator(BAMFileReader.java:959)
at htsjdk.samtools.BAMFileReader.query(BAMFileReader.java:612)
at htsjdk.samtools.SamReader$PrimitiveSamReaderToSamReaderAdapter.query(SamReader.java:533)
at htsjdk.samtools.SamReader$PrimitiveSamReaderToSamReaderAdapter.query(SamReader.java:538)
at htsjdk.samtools.SamReader$PrimitiveSamReaderToSamReaderAdapter.queryOverlapping(SamReader.java:400)
at com.astrazeneca.vardict.data.SamView.(SamView.java:21)
at com.astrazeneca.vardict.modules.VariationRealigner.noPassingReads(VariationRealigner.java:2060)
at com.astrazeneca.vardict.modules.VariationRealigner.realigndel(VariationRealigner.java:617)
at com.astrazeneca.vardict.modules.VariationRealigner.realignIndels(VariationRealigner.java:387)
at com.astrazeneca.vardict.modules.VariationRealigner.process(VariationRealigner.java:88)
at com.astrazeneca.vardict.modes.AbstractMode$$Lambda$43/0x00007ed9acccf458.apply(Unknown Source)
at java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:680)
at java.base/java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:658)
at java.base/java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:2094)
at com.astrazeneca.vardict.modes.AbstractMode.pipeline(AbstractMode.java:49)
at com.astrazeneca.vardict.modes.SomaticMode.processBothBamsInPipeline(SomaticMode.java:111)
at com.astrazeneca.vardict.modes.SomaticMode.access$000(SomaticMode.java:31)
at com.astrazeneca.vardict.modes.SomaticMode$SomdictWorker.call(SomaticMode.java:96)
at com.astrazeneca.vardict.modes.SomaticMode$SomdictWorker.call(SomaticMode.java:79)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
It is a bit puzzling to me having a "critical exception occurs on region: 2:1-59640629". In fact, this region was split in 10 kb overlapping windows.
Any suggestions?
Thanks in advance.
Best regards,
Lorenzo