search

jpmml / jpmml-sparkml

Java library and command-line application for converting Apache Spark ML pipelines to PMML
GNU Affero General Public License v3.0
267 stars 80 forks source link

java.lang.NoSuchMethodError on org.dmg.pmml.MiningField.setUsageType #12

Closed thetuxedo closed 7 years ago

thetuxedo commented 7 years ago

Dear Sir or Madam, When I try to export my model, I encounter following error. I am using the jpmml-sparkml 1.1.6, and spark 2.0.2

scala> val pmml = ConverterUtil.toPMML(df.schema, model)
java.lang.NoSuchMethodError: org.dmg.pmml.MiningField.setUsageType(Lorg/dmg/pmml/MiningField$UsageType;)Lorg/dmg/pmml/MiningField;
  at org.jpmml.converter.ModelUtil.createMiningField(ModelUtil.java:73)
  at org.jpmml.converter.ModelUtil.createMiningSchema(ModelUtil.java:57)
  at org.jpmml.converter.ModelUtil.createMiningSchema(ModelUtil.java:46)
  at org.jpmml.sparkml.model.RandomForestClassificationModelConverter.encodeModel(RandomForestClassificationModelConverter.java:45)
  at org.jpmml.sparkml.model.RandomForestClassificationModelConverter.encodeModel(RandomForestClassificationModelConverter.java:33)
  at org.jpmml.sparkml.ConverterUtil.toPMML(ConverterUtil.java:131)
  ... 52 elided

Thanks in advance for helping.

vruusmann commented 7 years ago

You need to configure your application classpath properly.

JPMML-SparkML version 1.1.6 is compiled against JPMML-Model version 1.3.4 (via JPMML-Converter version 1.2.1). However, Apache Spark version 2.0.X contains JPMML-Model version 1.2.15, which then "shadows" the proper version: https://issues.apache.org/jira/browse/SPARK-15526

Perhaps you should be using the JPMML-SparkML-Package library instead of the "raw" JPMML-Spark library? If you're determined to roll your own, then at least take a look at its pom.xml file for proper application packaging workflow: https://github.com/jpmml/jpmml-sparkml-package/blob/master/pom.xml#L39