search

apache / fury

A blazingly fast multi-language serialization framework powered by JIT and zero-copy.
https://fury.apache.org/
Apache License 2.0
3.08k stars 245 forks source link

deserialize failing with java.lang.OutOfMemoryError: Java heap space #1454

Open andyczerwonka opened 7 months ago

andyczerwonka commented 7 months ago

Search before asking

Version

latest 0.5.0 snapshot

Component(s)

Java/Scals

Minimal reproduce step

Here's the exception we're seeing.

org.apache.fury.exception.DeserializationException: Deserialize failed, read objects are: [PredictorStorageData(null,null,null), ConfigurationMetadata(Perfume SMILES price-process,Default Graph Model generated from data inspection.,null), Some(null), {}]
    at org.apache.fury.Fury.handleReadFailed(Fury.java:784)
    at org.apache.fury.Fury.deserialize(Fury.java:755)
    at org.apache.fury.Fury.deserialize(Fury.java:685)
    at org.apache.fury.ThreadLocalFury.deserialize(ThreadLocalFury.java:99)
    at io.citrine.mithril.modules.predictors.Predictor$.$anonfun$decode$1(Predictor.scala:111)
    at scala.util.Try$.apply(Try.scala:210)
    at io.citrine.mithril.modules.predictors.Predictor$.decode(Predictor.scala:107)
    at io.citrine.mithrilworkbench.Main$.$anonfun$tryToDecodeThisPredictor$1(Main.scala:330)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
    at scribe.LoggerSupport.elapsed(LoggerSupport.scala:67)
    at scribe.LoggerSupport.elapsed$(LoggerSupport.scala:62)
    at scribe.package$.elapsed(package.scala:8)
    at io.citrine.mithrilworkbench.Main$.tryToDecodeThisPredictor(Main.scala:323)
    at io.citrine.mithrilworkbench.Main$.$anonfun$main$5(Main.scala:340)
    at io.citrine.mithrilworkbench.Main$.$anonfun$main$5$adapted(Main.scala:340)
    at mainargs.Invoker$.$anonfun$invoke0$3(Invoker.scala:63)
    at mainargs.Result.flatMap(Result.scala:13)
    at mainargs.Result.flatMap$(Result.scala:12)
    at mainargs.Result$Success.flatMap(Result.scala:23)
    at mainargs.Invoker$.invoke0(Invoker.scala:62)
    at mainargs.Invoker$.invoke(Invoker.scala:72)
    at mainargs.Invoker$.$anonfun$runMains$1(Invoker.scala:92)
    at mainargs.Result.flatMap(Result.scala:13)
    at mainargs.Result.flatMap$(Result.scala:12)
    at mainargs.Result$Success.flatMap(Result.scala:23)
    at mainargs.Invoker$.groupArgs$1(Invoker.scala:92)
    at mainargs.Invoker$.runMains(Invoker.scala:107)
    at mainargs.ParserForMethods.runRaw0(Parser.scala:170)
    at mainargs.ParserForMethods.runEither(Parser.scala:108)
    at mainargs.ParserForMethods.runEither(Parser.scala:151)
    at mainargs.ParserForMethods.runOrExit(Parser.scala:56)
    at io.citrine.mithrilworkbench.Main$.main(Main.scala:341)
    at io.citrine.mithrilworkbench.Main.main(Main.scala)
Caused by: java.lang.OutOfMemoryError: Java heap space
    at java.base/java.lang.reflect.Array.multiNewArray(Native Method)
    at java.base/java.lang.reflect.Array.newInstance(Array.java:114)
    at org.apache.fury.serializer.ArraySerializers$ObjectArraySerializer.newArray(ArraySerializers.java:177)
    at org.apache.fury.serializer.ArraySerializers$ObjectArraySerializer.read(ArraySerializers.java:126)
    at org.apache.fury.serializer.ArraySerializers$ObjectArraySerializer.read(ArraySerializers.java:43)
    at org.apache.fury.Fury.readDataInternal(Fury.java:916)
    at org.apache.fury.Fury.readRef(Fury.java:831)
    at org.apache.fury.serializer.collection.AbstractMapSerializer.generalJavaRead(AbstractMapSerializer.java:568)
    at org.apache.fury.serializer.collection.AbstractMapSerializer.genericJavaRead(AbstractMapSerializer.java:443)
    at org.apache.fury.serializer.collection.AbstractMapSerializer.readElements(AbstractMapSerializer.java:435)
    at org.apache.fury.serializer.collection.MapSerializer.read(MapSerializer.java:46)
    at org.apache.fury.serializer.collection.MapSerializer.read(MapSerializer.java:27)
    at org.apache.fury.Fury.readDataInternal(Fury.java:916)
    at org.apache.fury.Fury.readRef(Fury.java:831)
    at org.apache.fury.serializer.ObjectSerializer.readOtherFieldValue(ObjectSerializer.java:438)
    at org.apache.fury.serializer.ObjectSerializer.readAndSetFields(ObjectSerializer.java:382)
    at org.apache.fury.serializer.ObjectSerializer.read(ObjectSerializer.java:316)
    at org.apache.fury.Fury.readDataInternal(Fury.java:916)
    at org.apache.fury.Fury.readRef(Fury.java:831)
    at org.apache.fury.serializer.ObjectSerializer.readOtherFieldValue(ObjectSerializer.java:438)
    at org.apache.fury.serializer.ObjectSerializer.readAndSetFields(ObjectSerializer.java:382)
    at org.apache.fury.serializer.ObjectSerializer.read(ObjectSerializer.java:316)
    at org.apache.fury.Fury.readDataInternal(Fury.java:916)
    at org.apache.fury.Fury.readRef(Fury.java:831)
    at org.apache.fury.serializer.ObjectSerializer.readOtherFieldValue(ObjectSerializer.java:438)
    at org.apache.fury.serializer.ObjectSerializer.readAndSetFields(ObjectSerializer.java:382)
    at org.apache.fury.serializer.ObjectSerializer.read(ObjectSerializer.java:316)
    at org.apache.fury.Fury.readDataInternal(Fury.java:916)
    at org.apache.fury.Fury.readRef(Fury.java:818)
    at org.apache.fury.Fury.deserialize(Fury.java:751)
    at org.apache.fury.Fury.deserialize(Fury.java:685)
    at org.apache.fury.ThreadLocalFury.deserialize(ThreadLocalFury.java:99)

What did you expect to see?

We're working with a 1G binary and we have 8G of heap space.

What did you see instead?

Caused by: java.lang.OutOfMemoryError: Java heap space
    at java.base/java.lang.reflect.Array.multiNewArray(Native Method)
    at java.base/java.lang.reflect.Array.newInstance(Array.java:114)
    at org.apache.fury.serializer.ArraySerializers$ObjectArraySerializer.newArray(ArraySerializers.java:177)

Anything Else?

No response

Are you willing to submit a PR?

chaokunyang commented 7 months ago

Hi @andyczerwonka , can I reproduce this error in my local environment?

chaokunyang commented 7 months ago

And could you provide your Fury creation code here too?

andyczerwonka commented 7 months ago

And could you provide your Fury creation code here too?

private val fury = Fury
    .builder()
    .withLanguage(Language.JAVA)
    .requireClassRegistration(false)
    .withScalaOptimizationEnabled(true)
    .withRefTracking(true)
    .withStringCompressed(true)
    .withLongCompressed(true)
    .withIntCompressed(true)
    .withAsyncCompilation(true)
    .buildThreadSafeFury()
andyczerwonka commented 7 months ago

Hi @andyczerwonka , can I reproduce this error in my local environment?

Unfortunately not, because the code is proprietary and I cannot share it. I was hoping the stack tracewould give us some insight into where it's failing so I could create a sample model.

chaokunyang commented 7 months ago

Could you disable sync compilation and try it again? There are some inconsistentency between compile mode and interpretor mode

andyczerwonka commented 7 months ago

Could you disable sync compilation and try it again? There are some inconsistentency between compile mode and interpretor mode

Assuming you want me to withAsyncCompilation(false), yes?

chaokunyang commented 7 months ago

Could you disable sync compilation and try it again? There are some inconsistentency between compile mode and interpretor mode

Assuming you want me to withAsyncCompilation(false), yes?

Yes, that's what I mean. Does it still fail?