Unable to save and load model

[prarshah]

I am using scala 2.11.8 and spark 2.4.0

Exception in thread "main" java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods$.parse(Lorg/json4s/JsonInput;Z)Lorg/json4s/JsonAST$JValue;
    at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelWriter$$anonfun$8.apply(IsolationForestModelReadWrite.scala:302)
    at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelWriter$$anonfun$8.apply(IsolationForestModelReadWrite.scala:301)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
    at scala.collection.AbstractTraversable.map(Traversable.scala:104)
    at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelWriter.getMetadataToSave(IsolationForestModelReadWrite.scala:301)
    at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelWriter.saveMetadata(IsolationForestModelReadWrite.scala:280)
    at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelWriter.saveImplHelper(IsolationForestModelReadWrite.scala:253)
    at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelWriter.saveImpl(IsolationForestModelReadWrite.scala:241)

[jverbus]

Are you building the project yourself? If so, are you specifying the sparkVersion in the build command?

./gradlew build -PsparkVersion=2.4.3

You can also download a Spark 2.4.3 jar here: https://bintray.com/linkedin/maven/isolation-forest/0.3.2#files/com%2Flinkedin%2Fisolation-forest%2Fisolation-forest_2.4.3_2.11%2F0.3.2

[prarshah]

Thank you for the quick reply, No I am using the maven dependency

com.linkedin.isolation-forest isolation-forest_2.3.0_2.11 0.3.2

When I changed the json4s-jackson's version to 3.2.11, it was able to save the model but was unable to read it.

[prarshah]

This is the updated error:

Exception in thread "main" java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods$.parse$default$3()Z at org.apache.spark.ml.param.IntParam.jsonDecode(params.scala:410) at org.apache.spark.ml.param.IntParam.jsonDecode(params.scala:390) at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelReader$Metadata$$anonfun$setParams$1.apply(IsolationForestModelReadWrite.scala:169) at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelReader$Metadata$$anonfun$setParams$1.apply(IsolationForestModelReadWrite.scala:167) at scala.collection.immutable.List.foreach(List.scala:392) at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelReader$Metadata.setParams(IsolationForestModelReadWrite.scala:167) at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelReader.load(IsolationForestModelReadWrite.scala:60) at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelReader.load(IsolationForestModelReadWrite.scala:38) at org.apache.spark.ml.util.MLReadable$class.load(ReadWrite.scala:380) at com.linkedin.relevance.isolationforest.IsolationForestModel$.load(IsolationForestModel.scala:140) at isolation_forest.OutlierDetectorEstimatorTest$.delayedEndpoint$isolation_forest$OutlierDetectorEstimatorTest$1(OutlierDetectorEstimatorTest.scala:60) at isolation_forest.OutlierDetectorEstimatorTest$delayedInit$body.apply(OutlierDetectorEstimatorTest.scala:9)

[jverbus]

That jar is built using Spark 2.3.0. Please try the jar built using 2.4.3 if you're using Spark 2.4.

https://search.maven.org/search?q=isolation-forest

or https://bintray.com/linkedin/maven/isolation-forest/0.3.2#files/com%2Flinkedin%2Fisolation-forest%2Fisolation-forest_2.4.3_2.11%2F0.3.2

[jverbus]

com.linkedin.isolation-forest isolation-forest_2.4.3_2.11 0.3.2