aidenlab / juicer

A One-Click System for Analyzing Loop-Resolution Hi-C Experiments
http://aidenlab.org
MIT License
418 stars 183 forks source link

addnorm: Requested array size exceeds VM limit #357

Open jiangshan529 opened 2 months ago

jiangshan529 commented 2 months ago

Hi, I am creating a hic file with 1.7 billion reads to 50 bp res map, but the addnorm always failed in the last step. How could this be resolved? I requested a google cloud server with 700gb memory and the used memory never exceeds 100g.

Calculating norms for zoom BP_50 Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit at java.base/java.util.Arrays.copyOf(Arrays.java:3537) at java.base/java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:100) at java.base/java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:111) at org.broad.igv.tdf.BufferedByteWriter.putLong(BufferedByteWriter.java:134) at org.broad.igv.tdf.BufferedByteWriter.putDouble(BufferedByteWriter.java:115) at juicebox.tools.utils.original.NormalizationVectorUpdater.writeNormalizationVector(NormalizationVectorUpdater.java:859) at juicebox.tools.utils.original.NormalizationVectorUpdater.updateHicFile(NormalizationVectorUpdater.java:174) at juicebox.tools.clt.old.AddNorm.run(AddNorm.java:84) at juicebox.tools.HiCTools.main(HiCTools.java:96)

moshe-olshansky commented 1 month ago

Hello, Which norm are you trying to add? KR may not work, but SCALE and its derivatives should.