search

jpmml / jpmml-sparkml

Java library and command-line application for converting Apache Spark ML pipelines to PMML
GNU Affero General Public License v3.0
267 stars 80 forks source link

Transformer class org.apache.spark.ml.feature.Mytransformer is not supported #77

Closed GuoYL36 closed 5 years ago

GuoYL36 commented 5 years ago

When I used the jpmml-spark library to generate a model in PMML format for pipeline model, this error appeared: Transformer class org. apache. spark. ml. feature. Mytransformer is not supported. Among them, Mytransformer operator is my own extended transformer operator, because org.apache.spark.ml supports developers to extend transformer operator, but jpmml-spark library does not support developer-defined transformer operator. What method can jpmml-spark support developers to customize operator?

vruusmann commented 5 years ago

Closing as exact duplicate of https://github.com/jpmml/jpmml-sparkml/issues/72

GuoYL36 commented 5 years ago

Closing as exact duplicate of #72

Thanks!

GuoYL36 commented 5 years ago

Closing as exact duplicate of #72 I don't understand you said that register it with the JPMML-SparkML runtime by mentioning it in some META-INF/sparkml2pmml.properties file: https://github.com/jpmml/jpmml-sparkml/blob/master/src/main/resources/META-INF/sparkml2pmml.properties#L1-L28. Do I only add "org.apache.spark.ml.feature.Mytransformer = org.jpmml.sparkml.feature.MytransformerConverter" in META-INF/sparkml2pmml.properties file, and then run my code?

vruusmann commented 5 years ago

I don't understand you said that register it with the JPMML-SparkML runtime by mentioning it in some META-INF/sparkml2pmml.properties file:

You should package your custom transformer class as a standalone JAR file, while contains its own META-INF/sparkml2pmml.properties file.

However, for quick and dirty hacking, you could put your custom transformer class into the official JPMML-SparkML codebase, and modify its META-INF/sparkml2pmml.properties file.

Do I only add "org.apache.spark.ml.feature.Mytransformer = org.jpmml.sparkml.feature.MytransformerConverter"

The format is:

<Apache Spark transformer class name> = <JPMML-SparkML transformer converter class name>