search

aidenlab / Juicebox

Visualization and analysis software for Hi-C data -
https://aidenlab.org/juicebox
MIT License
244 stars 58 forks source link

Memory Leak #146

Open sa501428 opened 9 years ago

sa501428 commented 9 years ago

screen shot 2015-06-15 at 7 22 13 pm

sa501428 commented 9 years ago

If any of you are particularly excited about going through the heap dump, let me know

nchernia commented 9 years ago

When does this happen? If it's during normalization it's not a memory leak, we need to hold all the contact records in memory (which is why it's expensive to normalize a dense matrix at high resolution)

On Tue, Jun 16, 2015 at 2:27 AM, Muhammad Saad Shamim < notifications@github.com> wrote:

If any of you are particularly excited about going through the heap dump, let me know

— Reply to this email directly or view it on GitHub https://github.com/theaidenlab/Juicebox/issues/146#issuecomment-112243776 .

Neva Cherniavsky Durand, Ph.D. Staff Scientist, Aiden Lab www.aidenlab.org

imachol commented 9 years ago

Does it get ‘cleaned’ later? It seems like a correct behavior, as Neva mentioned.

On Jun 16, 2015, at 1:23 AM, nchernia notifications@github.com<mailto:notifications@github.com> wrote:

When does this happen? If it's during normalization it's not a memory leak, we need to hold all the contact records in memory (which is why it's expensive to normalize a dense matrix at high resolution)

On Tue, Jun 16, 2015 at 2:27 AM, Muhammad Saad Shamim < notifications@github.commailto:notifications@github.com> wrote:

If any of you are particularly excited about going through the heap dump, let me know

— Reply to this email directly or view it on GitHub https://github.com/theaidenlab/Juicebox/issues/146#issuecomment-112243776 .

Neva Cherniavsky Durand, Ph.D. Staff Scientist, Aiden Lab www.aidenlab.orghttp://www.aidenlab.org

— Reply to this email directly or view it on GitHubhttps://github.com/theaidenlab/Juicebox/issues/146#issuecomment-112307954.

theaidenlab commented 9 years ago

This happens on APA for GM12878 and is not the correct behavior. I'll double check that the contributing error is not within the APA code. Otherwise I'll run APA without caching.

On Tue, Jun 16, 2015, 9:48 AM imachol notifications@github.com wrote:

Does it get ‘cleaned’ later? It seems like a correct behavior, as Neva mentioned.

On Jun 16, 2015, at 1:23 AM, nchernia <notifications@github.com<mailto: notifications@github.com>> wrote:

When does this happen? If it's during normalization it's not a memory leak, we need to hold all the contact records in memory (which is why it's expensive to normalize a dense matrix at high resolution)

On Tue, Jun 16, 2015 at 2:27 AM, Muhammad Saad Shamim < notifications@github.commailto:notifications@github.com> wrote:

If any of you are particularly excited about going through the heap dump, let me know

— Reply to this email directly or view it on GitHub < https://github.com/theaidenlab/Juicebox/issues/146#issuecomment-112243776>

.

Neva Cherniavsky Durand, Ph.D. Staff Scientist, Aiden Lab www.aidenlab.orghttp://www.aidenlab.org

— Reply to this email directly or view it on GitHub< https://github.com/theaidenlab/Juicebox/issues/146#issuecomment-112307954>.

— Reply to this email directly or view it on GitHub https://github.com/theaidenlab/Juicebox/issues/146#issuecomment-112456920 .

sa501428 commented 9 years ago

I think the following is related (was using Juicebox for about 30 minutes at multiple resolutions and several chromosomes):

/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/bin/java -Djava.library.path=/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/lwjgl:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jcuda -Xmx2000m -Djnlp.loadMenu=http://hicfiles.tc4ga.com/juicebox.properties -Didea.launcher.port=7533 "-Didea.launcher.bin.path=/Applications/IntelliJ IDEA 14 CE.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/tools.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/out/production/Juicebox:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jocl-java-src.zip:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-java-src.zip:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-linux-i586.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-linux-amd64.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-linux-armv6.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-linux-i586.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-solaris-i586.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-windows-i586.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-linux-amd64.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-linux-armv6.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-linux-armv6hf.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-solaris-amd64.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-windows-amd64.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-solaris-i586.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-windows-i586.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-linux-armv6hf.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-solaris-amd64.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-windows-amd64.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-macosx-universal.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-macosx-universal.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/lwjgl/lwjgl.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/broadinstitute/igv.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/broadinstitute/htsjdk-1.123.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jcuda/jcuda-0.7.0.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jcuda/jcudaUtils-0.0.4.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/general/jsi1.1.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/general/guava-18.0.jar:/Applications/IntelliJ IDEA 14 CE.app/Contents/lib/idea_rt.jar" com.intellij.rt.execution.application.AppMain juicebox.MainWindow Fetching user directory... Initializing Components INFO [2015-09-02 22:23:43,115] [DirectoryManager.java:149] [AWT-EventQueue-0] IGV Directory: /Users/muhammadsaadshamim/igv Window W: 1620 H1130 INFO [2015-09-02 22:23:49,445] [CommandListener.java:86] [Thread-5] Listening on port 30000 INFO [2015-09-02 22:23:56,520] [HttpUtils.java:809] [pool-1-thread-1] Range-byte request succeeded HiC file version: 7 2015-09-02 22:28:21.316 java[40113:3071811] Metadata.framework [Error]: couldn't get the client port 2015-09-02 22:28:21.874 java[40113:3069959] --ImageKit Error: _mirrorViewBlurImage: glPixelStorei error 502 2015-09-02 22:28:26.069 java[40113:3069959] --ImageKit Error: _mirrorViewBlurImage: glPixelStorei error 502 2015-09-02 22:28:27.083 java[40113:3069959] --ImageKit Error: _mirrorViewBlurImage: glPixelStorei error 502 INFO [2015-09-02 22:36:12,775] [GenomeManager.java:130] [pool-1-thread-1] Loading genome: http://igvdata.broadinstitute.org/genomes/hg19.genome INFO [2015-09-02 22:36:14,449] [GenomeManager.java:174] [pool-1-thread-1] Genome loaded. id= hg19 INFO [2015-09-02 22:36:37,786] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://hicfiles.s3.amazonaws.com/external/GM12878_RnaSeq_summed.bw INFO [2015-09-02 22:36:54,844] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878ControlStdSig.wig.tdf INFO [2015-09-02 22:36:54,849] [HttpUtils.java:837] [pool-1-thread-1] Testing range-byte request on host: igvdata.broadinstitute.org INFO [2015-09-02 22:36:55,844] [HttpUtils.java:809] [pool-1-thread-1] Range-byte request succeeded INFO [2015-09-02 22:36:56,587] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878CtcfStdSig.wig.tdf INFO [2015-09-02 22:36:57,720] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k27acStdSig.wig.tdf INFO [2015-09-02 22:36:59,868] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k27me3StdSig.wig.tdf INFO [2015-09-02 22:37:01,119] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k36me3StdSig.wig.tdf INFO [2015-09-02 22:37:02,280] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k4me1StdSig.wig.tdf INFO [2015-09-02 22:37:03,329] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k4me2StdSig.wig.tdf INFO [2015-09-02 22:37:04,699] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k4me3StdSig.wig.tdf INFO [2015-09-02 22:37:06,421] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igv.broadinstitute.org.s3.amazonaws.com/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k9me3StdSig.tdf INFO [2015-09-02 22:37:06,421] [HttpUtils.java:837] [pool-1-thread-1] Testing range-byte request on host: igv.broadinstitute.org.s3.amazonaws.com INFO [2015-09-02 22:37:07,034] [HttpUtils.java:809] [pool-1-thread-1] Range-byte request succeeded INFO [2015-09-02 22:37:07,963] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k9acStdSig.wig.tdf INFO [2015-09-02 22:37:09,419] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H4k20me1StdSig.wig.tdf 2015-09-02 22:56:22.083 java[40113:3069959] Lookup: Unhandled exception 'JavaNativeException' caught in +[LULookupDefinitionModule _focusTermUsingQueue:handler:] Exception in thread "Thread-210" java.lang.OutOfMemoryError: GC overhead limit exceeded at juicebox.data.DatasetReaderV2.readNormalizedBlock(DatasetReaderV2.java:691) at juicebox.data.MatrixZoomData$1.run(MatrixZoomData.java:216) at java.lang.Thread.run(Thread.java:745) Exception in thread "Thread-212" java.lang.OutOfMemoryError: GC overhead limit exceeded at juicebox.data.DatasetReaderV2.readNormalizedBlock(DatasetReaderV2.java:691) at juicebox.data.MatrixZoomData$1.run(MatrixZoomData.java:216) at java.lang.Thread.run(Thread.java:745) Exception in thread "Thread-213" java.lang.OutOfMemoryError: GC overhead limit exceeded at juicebox.data.DatasetReaderV2.readNormalizedBlock(DatasetReaderV2.java:691) at juicebox.data.MatrixZoomData$1.run(MatrixZoomData.java:216) at java.lang.Thread.run(Thread.java:745) Exception in thread "Thread-214" java.lang.OutOfMemoryError: GC overhead limit exceeded at juicebox.data.DatasetReaderV2.readNormalizedBlock(DatasetReaderV2.java:691) at juicebox.data.MatrixZoomData$1.run(MatrixZoomData.java:216) at java.lang.Thread.run(Thread.java:745) Exception in thread "Thread-216" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-215" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-217" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-218" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-219" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-220" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-221" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-222" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-224" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-223" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-225" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-226" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-227" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-230" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-229" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-231" java.lang.OutOfMemoryError: GC overhead limit exceeded

Process finished with exit code 143

nchernia commented 9 years ago

I've never seen this error:

2015-09-02 22:28:21.316 java[40113:3071811] Metadata.framework [Error]: couldn't get the client port 2015-09-02 22:28:21.874 java[40113:3069959] --ImageKit Error: _mirrorViewBlurImage: glPixelStorei error 502 2015-09-02 22:28:26.069 java[40113:3069959] --ImageKit Error: _mirrorViewBlurImage: glPixelStorei error 502 2015-09-02 22:28:27.083 java[40113:3069959] --ImageKit Error: _mirrorViewBlurImage: glPixelStorei error 502

How many tracks do you have loaded?

On Thu, Sep 3, 2015 at 12:19 AM, Muhammad Saad Shamim < notifications@github.com> wrote:

I think the following is related (was using Juicebox for about 30 minutes at multiple resolutions and several chromosomes):

/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/bin/java -Djava.library.path=/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/lwjgl:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jcuda -Xmx2000m -Djnlp.loadMenu=http://hicfiles.tc4ga.com/juicebox.properties -Didea.launcher.port=7533 "-Didea.launcher.bin.path=/Applications/IntelliJ IDEA 14 CE.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/tools.jar:/Library/Java/Ja vaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library /Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/out/production/Juicebox:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/ Juicebox5/lib/jogl/jocl-java-src.zip:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-java-src.zip:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-linux-i586.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-linux-amd64.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-linux-armv6.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-linux-i586.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-solaris-i586.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-windows-i586.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-linux-amd64.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-linux-armv6.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox 5/lib/jogl/jogl-all-natives-linux-armv6hf.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-solaris-amd64.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-windows-amd64.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-solaris-i586.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-windows-i586.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-linux-armv6hf.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-solaris-amd64.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-windows-amd64.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/jogl-all-natives-macosx-universal.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jogl/gluegen-rt-natives-macosx-universal.jar:/Users/m uhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/lwjgl/lwjgl.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/broadinstitute/igv.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/broadinstitute/htsjdk-1.123.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jcuda/jcuda-0.7.0.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/jcuda/jcudaUtils-0.0.4.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/general/jsi1.1.jar:/Users/muhammadsaadshamim/Documents/Github/Hydra/Juicebox5/lib/general/guava-18.0.jar:/Applications/IntelliJ IDEA 14 CE.app/Contents/lib/idea_rt.jar" com.intellij.rt.execution.application.AppMain juicebox.MainWindow Fetching user directory... Initializing Components INFO [2015-09-02 22:23:43,115] [DirectoryManager.java:149] [AWT-EventQueue-0] IGV Directory: /Users/muhammadsaadshamim/igv Window W: 1620 H1130 INFO [2015-09-02 22:23:49,445] [CommandListener.java:86] [Thread-5] Listening on port 30000 INFO [2015-09-02 22:23:56,520] [HttpUtils.java:809] [pool-1-thread-1] Range-byte request succeeded HiC file version: 7 2015-09-02 22:28:21.316 java[40113:3071811] Metadata.framework [Error]: couldn't get the client port 2015-09-02 22:28:21.874 java[40113:3069959] --ImageKit Error: _mirrorViewBlurImage: glPixelStorei error 502 2015-09-02 22:28:26.069 java[40113:3069959] --ImageKit Error: _mirrorViewBlurImage: glPixelStorei error 502 2015-09-02 22:28:27.083 java[40113:3069959] --ImageKit Error: _mirrorViewBlurImage: glPixelStorei error 502 INFO [2015-09-02 22:36:12,775] [GenomeManager.java:130] [pool-1-thread-1] Loading genome: http://igvdata.broadinstitute.org/genomes/hg19.genome INFO [2015-09-02 22:36:14,449] [GenomeManager.java:174] [pool-1-thread-1] Genome loaded. id= hg19 INFO [2015-09-02 22:36:37,786] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://hicfiles.s3.amazonaws.com/external/GM12878_RnaSeq_summed.bw INFO [2015-09-02 22:36:54,844] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878ControlStdSig.wig.tdf INFO [2015-09-02 22:36:54,849] [HttpUtils.java:837] [pool-1-thread-1] Testing range-byte request on host: igvdata.broadinstitute.org INFO [2015-09-02 22:36:55,844] [HttpUtils.java:809] [pool-1-thread-1] Range-byte request succeeded INFO [2015-09-02 22:36:56,587] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878CtcfStdSig.wig.tdf INFO [2015-09-02 22:36:57,720] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k27acStdSig.wig.tdf INFO [2015-09-02 22:36:59,868] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k27me3StdSig.wig.tdf INFO [2015-09-02 22:37:01,119] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k36me3StdSig.wig.tdf INFO [2015-09-02 22:37:02,280] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k4me1StdSig.wig.tdf INFO [2015-09-02 22:37:03,329] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k4me2StdSig.wig.tdf INFO [2015-09-02 22:37:04,699] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k4me3StdSig.wig.tdf INFO [2015-09-02 22:37:06,421] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igv.broadinstitute.org.s3.amazonaws.com/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k9me3StdSig.tdf INFO [2015-09-02 22:37:06,421] [HttpUtils.java:837] [pool-1-thread-1] Testing range-byte request on host: igv.broadinstitute.org.s3.amazonaws.com INFO [2015-09-02 22:37:07,034] [HttpUtils.java:809] [pool-1-thread-1] Range-byte request succeeded INFO [2015-09-02 22:37:07,963] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H3k9acStdSig.wig.tdf INFO [2015-09-02 22:37:09,419] [TrackLoader.java:112] [pool-1-thread-1] Loading resource, path http://igvdata.broadinstitute.org/data/hg19/encode/broadHistone/wgEncodeBroadHistoneGm12878H4k20me1StdSig.wig.tdf 2015-09-02 22:56:22.083 java[40113:3069959] Lookup: Unhandled exception 'JavaNativeException' caught in +[LULookupDefinitionModule _focusTermUsingQueue:handler:] Exception in thread "Thread-210" java.lang.OutOfMemoryError: GC overhead limit exceeded at juicebox.data.DatasetReaderV2.readNormalizedBlock(DatasetReaderV2.java:691) at juicebox.data.MatrixZoomData$1.run(MatrixZoomData.java:216) at java.lang.Thread.run(Thread.java:745) Exception in thread "Thread-212" java.lang.OutOfMemoryError: GC overhead limit exceeded at juicebox.data.DatasetReaderV2.readNormalizedBlock(DatasetReaderV2.java:691) at juicebox.data.MatrixZoomData$1.run(MatrixZoomData.java:216) at java.lang.Thread.run(Thread.java:745) Exception in thread "Thread-213" java.lang.OutOfMemoryError: GC overhead limit exceeded at juicebox.data.DatasetReaderV2.readNormalizedBlock(DatasetReaderV2.java:691) at juicebox.data.MatrixZoomData$1.run(MatrixZoomData.java:216) at java.lang.Thread.run(Thread.java:745) Exception in thread "Thread-214" java.lang.OutOfMemoryError: GC overhead limit exceeded at juicebox.data.DatasetReaderV2.readNormalizedBlock(DatasetReaderV2.java:691) at juicebox.data.MatrixZoomData$1.run(MatrixZoomData.java:216) at java.lang.Thread.run(Thread.java:745) Exception in thread "Thread-216" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-215" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-217" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-218" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-219" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-220" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-221" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-222" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-224" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-223" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-225" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-226" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-227" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-230" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-229" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "Thread-231" java.lang.OutOfMemoryError: GC overhead limit exceeded

Process finished with exit code 143

— Reply to this email directly or view it on GitHub https://github.com/theaidenlab/Juicebox/issues/146#issuecomment-137326579 .

Neva Cherniavsky Durand, Ph.D. Staff Scientist, Aiden Lab www.aidenlab.org

nchernia commented 8 years ago

Reenable caching and run APA and this bug will show up

imachol commented 8 years ago

I need more information about this bug - how do you enable cache? How did you run APA that fails?

How is it connected to Juicebox viewer?

nchernia commented 8 years ago

Search for HiCGlobals.useCache

It's turned on in regular Juicebox and turned off in HiCTools with a TODO next to it.

My understanding is that if you turn it on in HiCTools, APA will fail.

Connected to Juicebox viewer in that it's turned on and we think that it's causing a memory leak (that shows up when you play with Juicebox for a while).

On Fri, Dec 18, 2015 at 12:56 PM, imachol notifications@github.com wrote:

I need more information about this bug - how do you enable cache? How did you run APA that fails?

How is it connected to Juicebox viewer?

— Reply to this email directly or view it on GitHub https://github.com/theaidenlab/Juicebox/issues/146#issuecomment-165855746 .

Neva Cherniavsky Durand, Ph.D. Staff Scientist, Aiden Lab www.aidenlab.org

nchernia commented 8 years ago

Here's my take on this.

Possible solution (via Jim): Add some code that clears caches when you jump somewhere else, for example visit another chromosome, or change zoom levels. The caching is probably most important for supporting panning, for other non-local jumps you could flush them. There will be a delay of course if you return to the same location as it will have to load again. This would require a bit of coding, but not too much. Most of the navigation methods go through the HiC class, you can catch non-local jumps there. You could also just delete the existing "MatrixZoomData" object, I think its loaded on demand if not there.