search

rrwick / Unicycler

hybrid assembly pipeline for bacterial genomes
GNU General Public License v3.0
535 stars 132 forks source link

pilon out of memory error #330

Open DenisGoryunov opened 6 months ago

DenisGoryunov commented 6 months ago

Hi, I got pilon error and i'm wondering, how i can fix it. Could you help me please? Can i solve it by setting up environmental variables:

export _JAVA_OPTIONS="-Xms512m -Xmx32g"

Regards, Denis

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at java.base/java.util.Arrays.copyOfRange(Arrays.java:4030)
        at java.base/java.lang.StringLatin1.newString(StringLatin1.java:715)
        at java.base/java.lang.String.substring(String.java:1879)
        at scala.collection.StringOps$.slice$extension(StringOps.scala:627)
        at org.broadinstitute.pilon.Assembler.$anonfun$addToPileups$1(Assembler.scala:81)
        at org.broadinstitute.pilon.Assembler$$Lambda$104/0x00000001001be840.apply$mcVI$sp(Unknown Source)
        at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:190)
        at org.broadinstitute.pilon.Assembler.addToPileups(Assembler.scala:80)
        at org.broadinstitute.pilon.Assembler.addRead(Assembler.scala:69)
        at org.broadinstitute.pilon.Assembler.$anonfun$addReads$1(Assembler.scala:47)
        at org.broadinstitute.pilon.Assembler.$anonfun$addReads$1$adapted(Assembler.scala:47)
        at org.broadinstitute.pilon.Assembler$$Lambda$103/0x00000001001bd840.apply(Unknown Source)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at org.broadinstitute.pilon.Assembler.addReads(Assembler.scala:47)
        at org.broadinstitute.pilon.GapFiller.assembleIntoBreak(GapFiller.scala:127)
        at org.broadinstitute.pilon.GapFiller.assembleAcrossBreak(GapFiller.scala:55)
        at org.broadinstitute.pilon.GapFiller.fixBreak(GapFiller.scala:46)
        at org.broadinstitute.pilon.GenomeRegion.$anonfun$identifyAndFixIssues$6(GenomeRegion.scala:401)
        at org.broadinstitute.pilon.GenomeRegion.$anonfun$identifyAndFixIssues$6$adapted(GenomeRegion.scala:399)
        at org.broadinstitute.pilon.GenomeRegion$$Lambda$94/0x00000001001b7840.apply(Unknown Source)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at org.broadinstitute.pilon.GenomeRegion.identifyAndFixIssues(GenomeRegion.scala:399)
        at org.broadinstitute.pilon.GenomeFile.$anonfun$processRegions$4(GenomeFile.scala:113)
        at org.broadinstitute.pilon.GenomeFile.$anonfun$processRegions$4$adapted(GenomeFile.scala:102)
        at org.broadinstitute.pilon.GenomeFile$$Lambda$51/0x000000010016f840.apply(Unknown Source)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at org.broadinstitute.pilon.GenomeFile.processRegions(GenomeFile.scala:102)
        at org.broadinstitute.pilon.Pilon$.main(Pilon.scala:111)
        at org.broadinstitute.pilon.Pilon.main(Pilon.scala)
danielpeterson0530 commented 6 months ago

I had a similar/the same issue (see error below) and was able to fix it by changing the options you mentioned, but I did it in pilon itself.

Opening pilon in a text editor and changing Line 16: default_jvm_mem_opts = ['-Xms512m', '-Xmx1g'] to default_jvm_mem_opts = ['-Xms512m', '-Xmx360g'] (360g being the max amount of RAM I allocated to Java/Pilon) worked for me.

You can quickly find where pilon is installed using the command: whereis pilon

It would seem that if not declared (and I could not find any -xmx flags passed via the Unicycler python scripts for pilon), the java runtime env is limited by default by pilon to a max 1g of RAM, which is likely not enough.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at java.base/java.lang.Object.clone(Native Method)
        at org.broadinstitute.pilon.NormalDistribution.<init>(NormalDistribution.scala:31)
        at org.broadinstitute.pilon.NormalDistribution.<init>(NormalDistribution.scala:56)
        at org.broadinstitute.pilon.GenomeRegion.fragCoverageDist$lzycompute(GenomeRegion.scala:95)
        at org.broadinstitute.pilon.GenomeRegion.fragCoverageDist(GenomeRegion.scala:95)
        at org.broadinstitute.pilon.GenomeRegion.postProcess(GenomeRegion.scala:275)
        at org.broadinstitute.pilon.GenomeFile.$anonfun$processRegions$4(GenomeFile.scala:106)
        at org.broadinstitute.pilon.GenomeFile.$anonfun$processRegions$4$adapted(GenomeFile.scala:102)
        at org.broadinstitute.pilon.GenomeFile$$Lambda$41/0x0000000100160840.apply(Unknown Source)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at org.broadinstitute.pilon.GenomeFile.processRegions(GenomeFile.scala:102)
        at org.broadinstitute.pilon.Pilon$.main(Pilon.scala:111)
        at org.broadinstitute.pilon.Pilon.main(Pilon.scala)
DenisGoryunov commented 6 months ago

Hi, Thank you so much for your detailed response! It's very helpful. Regards, Denis