Closed Ben-Epstein closed 3 years ago
@Ben-Epstein is it possible to send me some sample data so that I can try this out?
Or if you could please send the serialized zip, that would be good as well.
I've actually followed the mleap demo for pyspark https://github.com/ancasarb/mleap-demo/blob/master/notebooks/PySpark%20-%20AirBnb.ipynb, serialized the pipeline and re-loaded it in Scala as below
val pipeline = (for (bf <- managed(BundleFile(file))) yield {
bf.loadMleapBundle().get.root
}).tried.get.asInstanceOf[Pipeline]
println(pipeline)
pipeline.inputSchema.fields.zipWithIndex.foreach { case (field, idx) =>
println(s"$idx $field")
}
val frame = FrameReader().read(new File(getClass.getClassLoader.getResource("frame.airbnb.json").getFile)).get
val result = pipeline.transform(frame)
println(result.get)
Using the frame.airbnb.json file for example
"schema": {
"fields": [{
"name": "state",
"type": "string"
}, {
"name": "bathrooms",
"type": "double"
}, {
"name": "square_feet",
"type": "double"
}, {
"name": "bedrooms",
"type": "double"
}, {
"name": "security_deposit",
"type": "double"
}, {
"name": "cleaning_fee",
"type": "double"
}, {
"name": "extra_people",
"type": "double"
}, {
"name": "number_of_reviews",
"type": "double"
}, {
"name": "review_scores_rating",
"type": "double"
}, {
"name": "room_type",
"type": "string"
}, {
"name": "host_is_superhost",
"type": "string"
}, {
"name": "cancellation_policy",
"type": "string"
}, {
"name": "instant_bookable",
"type": "string"
}]
},
"rows": [["NY", 2.0, 1250.0, 3.0, 50.0, 30.0, 2.0, 56.0, 90.0, "Entire home/apt", "1.0", "strict", "1.0"]]
}
and everything worked fine, am I missing anything?
How are you doing the prediction?
@Ben-Epstein is it possible to send me some sample data so that I can try this out?
@ancasarb the dataset is in the code linked above. It's the load_breast_cancer() dataset from sklearn.
Let me know if you need any help!
Is there any update on this?
@Ben-Epstein Looking at the code for the pipeline schema here, it does seem that we use a hashmap to collect the schema for the pipeline, so it could be that the order is different. However, this shouldn't be an issue at scoring. In the serialized vector assembler, I can see
{
"name": "VectorAssembler_48d5abe041b949877a36",
"shape": {
"inputs": [{
"port": "input0",
"name": "mean radius"
}, {
"port": "input1",
"name": "mean texture"
}, {
"port": "input2",
"name": "mean perimeter"
}, {
"port": "input3",
"name": "mean area"
}, {
"port": "input4",
"name": "mean smoothness"
}, {
"port": "input5",
"name": "mean compactness"
}, {
"port": "input6",
"name": "mean concavity"
}, {
"port": "input7",
"name": "mean concave points"
}, {
"port": "input8",
"name": "mean symmetry"
}, {
"port": "input9",
"name": "mean fractal dimension"
}, {
"port": "input10",
"name": "radius error"
}, {
"port": "input11",
"name": "texture error"
}, {
"port": "input12",
"name": "perimeter error"
}, {
"port": "input13",
"name": "area error"
}, {
"port": "input14",
"name": "smoothness error"
}, {
"port": "input15",
"name": "compactness error"
}, {
"port": "input16",
"name": "concavity error"
}, {
"port": "input17",
"name": "concave points error"
}, {
"port": "input18",
"name": "symmetry error"
}, {
"port": "input19",
"name": "fractal dimension error"
}, {
"port": "input20",
"name": "worst radius"
}, {
"port": "input21",
"name": "worst texture"
}, {
"port": "input22",
"name": "worst perimeter"
}, {
"port": "input23",
"name": "worst area"
}, {
"port": "input24",
"name": "worst smoothness"
}, {
"port": "input25",
"name": "worst compactness"
}, {
"port": "input26",
"name": "worst concavity"
}, {
"port": "input27",
"name": "worst concave points"
}, {
"port": "input28",
"name": "worst symmetry"
}, {
"port": "input29",
"name": "worst fractal dimension"
}],
"outputs": [{
"port": "output",
"name": "features"
}]
}
}
so the order is correct.
If then create a leap frame
"schema": {
"fields": [{
"name": "mean texture",
"type": "double"
}, {
"name": "concavity error",
"type": "double"
},
.....
]
},
"rows": [[23.95, 0.05051, ....]]
}
then the model scores fine, even if the order that we provide in the json is different than the one from the schema, or the ordering of the columns in the training data. As long as the "fields" and "rows" in the json respect the same ordering (any ordering) the scoring will be correct.
Hope this helps, let me know if you have further questions.
Closing this in preparation of 0.16.0 release, please reopen if you still have questions, hopefully the clarification above makes sense.
@ancasarb thank you for checking the code, however I'm still getting incorrect predictions with the code above. Did you run it and confirm that predictions on the MLeap transformer match the original spark model? All of the code necessary is above.
@Ben-Epstein Yes, I ran it without issues. Could you please share a leap frame that you use for the scoring please?
Worked fine for me with the latest version ml.combust.mleap:mleap-spark_2.11:0.16.0
closing this, please re-open if you're still struggling with it.
After creating a PySpark model and serializing it to a bundle, I try to read in the MLeap transformer and make a prediction but the prediction is wrong.
Upon investigation, I've found that the inputSchema of the model has been modified, so the features are in the wrong order. If you simply print out the PipelineModel, it shows the features in the correct order, but calling inputSchema gives an incorrect order. @abaveja313 ** Code to reproduce:
Model
serialization and movement to HDFS:
Reading in the model
printing the inputSchema:
As you can see, the inputSchema is wrong, causing all predictions to be wrong. I've reproduced the same with LogisticRegression models as well. I'm stuck here because without being able to generate the schema I have to specify it each time which creates non-reproducible code.
Is there something I'm doing wrong here or missing? Help would be greatly appreciated!