Closed ademasi closed 5 years ago
The Android OS doesn't provide/support a JAXB runtime. Even if you add org.jpmml:pmml-model-metro
dependency manually (ie. a full-blown Glassfish Metro JAXB runtime), then it can't be (class-)loaded in your Android application.
Forget about transmitting your PMML models as XML documents.
You have two viable technical options:
Dear Villu,
Thanks for your fast reply. I understand that Android OS does not have a JAXB runtime. I thought that the maven plugin doing the conversion to old Java serialization data format needed it anyway.
Concerning the options :
I tried to convert the PMML file created by JPMML-SkLearn to JSON using org.jpmml.model.TranslationExample from the pmml-model-example repository. It failed with a weird error :
alex@daedalus ~/g/j/pmml-model-example> java -cp target/example-1.4-SNAPSHOT.jar org.jpmml.model.TranslationExample --input ../../QoE.pmml --output QoE.pmml.json
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.UnsupportedOperationException) (through reference chain: org.dmg.pmml.PMML["Model"]->java.util.ArrayList[0]->org.dmg.pmml.mining.MiningModel["Segmentation"]->org.dmg.pmml.mining.Segmentation["Segment"]->java.util.ArrayList[0]->org.dmg.pmml.mining.Segment["Model"]->org.dmg.pmml.mining.MiningModel["Segmentation"]->org.dmg.pmml.mining.Segmentation["Segment"]->java.util.ArrayList[0]->org.dmg.pmml.mining.Segment["Model"]->org.dmg.pmml.tree.TreeModel["Node"]->org.dmg.pmml.tree.BranchNode["nodes"]->java.util.ArrayList[0]->org.dmg.pmml.tree.BranchNode["nodes"]->java.util.ArrayList[0]->org.dmg.pmml.tree.BranchNode["nodes"]->java.util.ArrayList[0]->org.dmg.pmml.tree.BranchNode["nodes"]->java.util.ArrayList[0]->org.dmg.pmml.tree.BranchNode["nodes"]->java.util.ArrayList[0]->org.dmg.pmml.tree.LeafNode["extensions"])
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:394)
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:353)
at com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:316)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:727)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSerializerBase.java:604)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:729)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSerializerBase.java:604)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:729)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSerializerBase.java:604)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:729)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSerializerBase.java:604)
at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:147)
at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107)
at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1120)
at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:950)
at org.jpmml.model.TranslationExample.execute(TranslationExample.java:83)
at org.jpmml.model.Example.execute(Example.java:44)
at org.jpmml.model.TranslationExample.main(TranslationExample.java:44)
Caused by: java.lang.UnsupportedOperationException
at org.dmg.pmml.tree.Node.getExtensions(Node.java:64)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:688)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
... 72 more
The model is just a standard scaler followed by a XGBoost classifier.
I thought the convertion to .ser data format was available via this project. I pull this repository again, changed the android sdk version to 28 (last available on my machine) and run maven. But the same error happen again, as in my first message. Hence I am quite lost.
Thank you for your time.
Caused by: java.lang.UnsupportedOperationException at org.dmg.pmml.tree.Node.getExtensions(Node.java:64)
For some reason Jackson is trying to get the state of an object (here, an org.dmg.pmml.tree.BranchNode
object) using an accessor method, when it only should be interacting with object fields.
This issue could be related to https://github.com/jpmml/jpmml-model/issues/21
Perhaps polymorphic org.dmg.pmml.tree.Node
subclasses are missing some EclipseLink and Jackson-specific annotations (the stuff works with Glassfish Metro JAXB runtime, which is the gold standard for me).
As a quick workaround, you could try "rewriting" the PMML class model object by replacing polymorphic Node
subclasses with the default org.dmg.pmml.tree.ComplexNode
class.
I wonder if the Jackson serialization works with models that are not decision tree based. For example, do RegressionModel
elements work or not?
I tried with a StantardScaler and a LinearRegression classifier, the conversion went well.
What do you mean by "rewriting" the PMML class model object with ComplexNode ? This one ?
I checked my folders and before the conversion crash, it is able to generate a part of the json.
I can provide the .pmml file if needeed.
If it is related to jpmml/jpmml-model#21 , would change moxy to glassfish solve the issue ?
What do you mean by "rewriting" the PMML class model object with ComplexNode ?
In pseudocode:
TreeModel treeModel = ...;
for(Iterator<Node> nodeIt = treeModel.getAllNodes(); nodeIt.hasNext(); ){
Node node = nodeIt.next();
if(!(node instanceof ComplexNode)){
node = new ComplexNode(node);
nodeIt.set(node);
}
}
What do you mean by "rewriting" the PMML class model object with ComplexNode ?
Alternatively:
org.dmg.pmml.adapters.NodeAdapter
implementation (by setting the NodeAdapter#NODE_TRANSFORMER_PROVIDER
thread local object) which keeps all Node
instances as org.dmg.pmml.tree.ComplexNode
objects. The default NodeAdapter
implementation attempts to choose the most memory-efficient Node
subclass, which apparently causes problems for EclipseLink and Jackson.Node
subclasses in use, everything should be OK.Thank you for your advises.
I have been trying for the past day, without any success, to implement a solution. I tried different approaches.
I was able to serialize the model (.ser), but once I try to load the model on the Android device, an error happened, BranchNode is not available on the version pmml-evaluator:1.4.8 with pmml-model: 1.4.10. Those versions where the last available ones that did not generate this error Failed
resolution of: Ljavax/xml/bind/annotation/adapters/XmlAdapter` (e.g. with pmml-evaluator:1.4.11 and pmml-model:1.4.13).
I tried to switch to json, I took back the simple StantardScaler and LinearRegression classifier from my early test and convert it. I then tried to open it with the EvaluatorUtil.createEvaluator method to no success as it expect something else that a json file. I don't really understand how to load json pmml model. From what I understood pmml-evaluator always expect a pmml file, I did not find any example of loading a json model to create an Evaluator.
Option : NodeAdapter I have been looking for setting up a custom NodeAdapter. I was not able to find a way of doing that during the unmarshalling of PMML file.
Option : Rewriting with ComplexeNode I was able to load the pmml file, take the model from it, but I end up the instance of MiningModel that I can not cast as a TreeModel. Looking more into it, I should get to the MiningModel, the Segmention and a list of Segments. Once I have the list, I iterate on all the TreeModel that it contains.
Again, thank you for your help, it has been fascinating to learn about the PMML format, the jpmml project and your work on it.
@SheepOnMeth JSON serialization was failing because of missing Jackson annotations on polymorphic Node
subclasses. This has been fixed in JPMML-Model trunk (ie. master
branch). A new version should be released within ~one week, after I have completed some other things that are on my mind there.
You should be able to ship a model as JSON or YAML file from desktop computer to Android now.
In Android, simply parse this JSON or YAML file to org.dmg.pmml.PMML
instance using regular Jakcon tools such as ObjectMapper
. Then, construct an Evaluator instance using ModelEvaluatorBuilder
as specified in the README file of the JPMML-Evaluator project.
This discussion has given me the idea that the JPMML-Evaluator should be more "welcoming" to alternative data formats such as JSON and YAML (see the issue linked above).
@vruusmann Thanks, I was able to pull the new pmml-model and convert my model to json.
I update pmml-evaluator dependency to get the SNAPSHOT pmml-model version that I assembled on my machine. I recompile the android library using this updated evaluator. But I discover that the minimize version of the jar removed the ModelEvaluatorBuilder class. I disable minimize and compile the android library that contain ModelEvaluatorBuilder with success.
I put the model on the Android smartphone and I loaded it with this Kotlin code :
@Throws(Exception::class)
private fun createEvaluator(modeleName:String): Evaluator {
val assetManager = assets
val ims : InputStream = assetManager.open(modeleName)
val reader : Reader = InputStreamReader(ims)
val jsonPmml : PMML =Gson().fromJson(reader, PMML::class.java)
val modelEvaluatorBuilder : EvaluatorBuilder = ModelEvaluatorBuilder(jsonPmml)
return modelEvaluatorBuilder.build()
}
Jackson did not work when I tried to load the json to a PMML object. It was excepting java classes that Android JVM does not have, hence I switched to Gson. But I failed to get it working, when it tried to generate the PMML object, I obtain this issue.. From what I can see, I am able to access some part of the PMML object, but it crashed when it tried to use findModel from PMMLUtil.
Hi,
Thanks for all your work on the jpmml ecosystem. I was able to learn a lot.
I have created a PMML model from a PMML pipeline with jpmml-sklearn. I was able to load it and test in my Python environment without any issue. I am now trying to ship it in a Android application.
I have followed the example in the pom.xml file in the jpmml-android-exemple directory (I have generated android jar library), but every time I try to convert the model I end up with this error :
Since it is a jabx issue and I am running Java 11, I went and try to run the jabx_demo project.. But the build was successful, hence I think the issue is not from my JVM.
I have upgraded the pmml-model library to the last available version, added the minimal glassfish metro dependency and customize the pom file to only do the conversion in a specific directory. The file is available here.
Please, if you have any idea from where this error is from, I would be happy to hunt the bug down.
Regards, Alex.