search

onnx / onnxmltools

ONNXMLTools enables conversion of models to ONNX
https://onnx.ai
Apache License 2.0
992 stars 180 forks source link

KeyError: 'pyspark.ml.regression.IsotonicRegressionModel' when trying to convert Pyspark model to ONNX format #618

Open shyy5358 opened 1 year ago

shyy5358 commented 1 year ago

I am converting a pyspark trained model to ONNX format. More specifically, it's the pyspark.ml.regression.IsotonicRegressionModel. I get an error saying KeyError: 'pyspark.ml.regression.IsotonicRegressionModel'.

This is the piece of code I use:

initial_types = buildInitialTypesSimple(df.select("score"))
    onnx_model = convert_sparkml(cal_model, 'Pyspark model', initial_types, spark_session = spark)
with open(os.path.join("onnxmodels/", "model.onnx"), "wb") as f:
        f.write(onnx_model.SerializeToString()

Based on my understanding of the codebase, I believe currently the isotonic regression model is not yet supported by ONNX conversion. Is there any workaround for this? and what will be the plan/timeline to include the isotonic regression pyspark model conversion in the source codebase? Thank you

xadupre commented 1 year ago

The converter is usually added to sklearn-onnx because the development is easier, then ported to onnxmltools. However, I don't have any timeline I can share today.