search

jpmml / jpmml-evaluator-spark

PMML evaluator library for the Apache Spark cluster computing system (http://spark.apache.org/)
GNU Affero General Public License v3.0
94 stars 43 forks source link

AbstractMethodError: org.shaded.jpmml.evaluator.spark.PMMLTransformer #14

Closed wobblyknees closed 6 years ago

wobblyknees commented 6 years ago

I am getting the above exception when invoking the transform() method. I've shaded my jar (incl. relocation). Using Spark 2, jpmml-evaluator-spaprk "1.1-SNAPSHOT" version. I'm passing in a dataframe to transform(). Did I miss something?

Here's my code: case class Attribute(ATTR1: Double, ATTR2:Double,ATTR3:Double, ATTR4:Double) val ds = spark.read.option("header", "true").schema(StructType(Array(StructField("ATTR1", DoubleType, true),StructField("ATTR2", DoubleType, true),StructField("ATTR3", DoubleType, true),StructField("ATTR4", DoubleType, true)))).csv("my.csv").as[Attribute]

val file = new File(”mypmml.xml")
val evaluator = org.jpmml.evaluator.spark.EvaluatorUtil.createEvaluator(file)

val pmmlTransformerBuilder = new org.jpmml.evaluator.spark.TransformerBuilder(evaluator).withOutputCols
val pmmlTransformer = pmmlTransformerBuilder.build()
val output = pmmlTransformer.transform(ds)   // exception triggered here:

I get an exception on the last line:

Exception in thread "main" java.lang.AbstractMethodError: org.shaded.jpmml.evaluator.spark.PMMLTransformer.transform(Lorg/apache/spark/sql/Dataset;)Lorg/apache/spark/sql/Dataset; at org.equifax.spark.pmml.examples.TestPMML$.main(TestPMML.scala:52) at org.equifax.spark.pmml.examples.TestPMML.main(TestPMML.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:755) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

wobblyknees commented 6 years ago

I compiled the project with my own code and got past this issue.