jpmml / jpmml-transpiler

Java Transpiler (Translator + Compiler) API for PMML
GNU Affero General Public License v3.0
28 stars 2 forks source link

The `Classification(Type, ValueMap)` constructor is invisible #8

Closed rhorrell closed 4 years ago

rhorrell commented 4 years ago

Hi, I am getting error error on a very large model PMML (178MB)

/PMML$1583159071.java:448333: error: <K,V>Classification(Type,ValueMap<K,V>) has protected access in Classification
            return new Classification<>(Classification.Type.VOTE, values);
                   ^
  where K,V are type-variables:
    K extends Object declared in class Classification
    V extends Number declared in class Classification
1 error
java.io.IOException
        at org.jpmml.codemodel.CompilerUtil.compile(CompilerUtil.java:81)
        at org.jpmml.codemodel.CompilerUtil.compile(CompilerUtil.java:56)
        at org.jpmml.codemodel.CompilerUtil.compile(CompilerUtil.java:49)
        at org.jpmml.transpiler.TranspilerUtil.compile(TranspilerUtil.java:75)
        at org.jpmml.transpiler.Main.run(Main.java:115)
        at org.jpmml.transpiler.Main.main(Main.java:98)

Thoughts? Suggestions?

vruusmann commented 4 years ago

It doesn't seem to be related to the size of the PMML file - the compiler is complaining about an inaccessible type (too restrictive access modifiers), not about the size of a method/class.

Very difficult for me to say more based on this IOException. You should open the file PMML$1583159071.java in a regular IDE (eg. Eclipse) and see if IDE reports a similar issue (inaccessible type) or not.

My integration tests are running file on Java 1.8 (aka Java 8). Perhaps you're using some different Java version or a non-JDK Java compiler.

vruusmann commented 4 years ago

Or perhaps your JPMML-Evaluator and JPMML-Transpiler versions are mismatching. Right now, you should be working with JPMML-Transpiler 1.1-SNAPSHOT, which depends on JPMML-Evaluator 1.5.0.

rhorrell commented 4 years ago

I am able to reproduce the error using a example model on the DMG site http://dmg.org/pmml/pmml_examples/rattle_pmml_examples/AuditTree.xml

I just pulled down the JPMML-Evaluator and JPMML-Transpiler in the past few days.

vruusmann commented 4 years ago

I am able to reproduce the error using a example model on the DMG site.

The examples provided by DMG.org are outdated/full of garbage.

This issue is a compiler exception about an inaccessible constructor. It manifests itself when transpiling (multi-class-)classification models, which invoke this specific ValueMap constructor.

I just pulled down the JPMML-Evaluator and JPMML-Transpiler in the past few days.

Just pull the JPMML-Transpiler library again. Right now you're pairing JPMML-Transpiler 1.0.4 with JPMML-Evaluator 1.5.0, and these two are not source compatible.

This is the offending commit: https://github.com/jpmml/jpmml-evaluator/commit/0248d8f37b67f43882ea0b156b7d0728bd944653

rhorrell commented 4 years ago

Sorry about that with the DMG model. I was able to get permission to send you a mocked up version of the PMML (see attached)

test.pmml.txt

Here is what I am getting with a fresh git clone and build:

$ java -jar target/jpmml-transpiler-executable-1.1-SNAPSHOT.jar --xml-input test.pmml --jar-output test.jar /PMML$2003891312.java:521: error: <K,V>Classification(Type,ValueMap<K,V>) has protected access in Classification return new Classification<>(Classification.Type.VOTE, values); ^ where K,V are type-variables: K extends Object declared in class Classification V extends Number declared in class Classification 1 error java.io.IOException at org.jpmml.codemodel.CompilerUtil.compile(CompilerUtil.java:81) at org.jpmml.codemodel.CompilerUtil.compile(CompilerUtil.java:56) at org.jpmml.codemodel.CompilerUtil.compile(CompilerUtil.java:49) at org.jpmml.transpiler.TranspilerUtil.compile(TranspilerUtil.java:75) at org.jpmml.transpiler.Main.run(Main.java:115) at org.jpmml.transpiler.Main.main(Main.java:98)

$ java -version openjdk version "1.8.0_252" OpenJDK Runtime Environment Corretto-8.252.09.1 (build 1.8.0_252-b09) OpenJDK 64-Bit Server VM Corretto-8.252.09.1 (build 25.252-b09, mixed mode)

vruusmann commented 4 years ago

My attempt to transpile your sample PMML file with the newly released JPMML-Transpiler 1.1.0:

$ java -jar jpmml-transpiler-executable-1.1.0.jar --input test.pmml.txt --output test.jar
/PMML$221634215.java:669: error: <K,V>Classification(Type,ValueMap<K,V>) has protected access in Classification
            return new Classification<>(Classification.Type.VOTE, values);
                   ^
  where K,V are type-variables:
    K extends Object declared in class Classification
    V extends Number declared in class Classification
1 error

OK - Taking my previous words back - the JPMML-Transpiler version 1.1.0 is generating invalid Java source code at the moment.

As a first step, will add a command-line switch to preserve the incomplete Service JAR file (only the problematic source file, no class files) to make it possible to inspect and import it into IDE.

vruusmann commented 4 years ago

Hoping to find time to troubleshoot this issue over the weekend, not sooner.

rhorrell commented 4 years ago

Hoping to find time to troubleshoot this issue over the weekend, not sooner.

Thank you very much!!!!

vruusmann commented 4 years ago

The easy fix would be to generate new Classification<>(Classification.Type.VOTE, values){} (note the empty pair of curly braces!) in such a place - a subclass can see protected constructors of the parent class.

The real fix would involve relaxing these access modifiers from protected back to public in the JPMML-Evaluator library. Why did I restrict it in the first place?

My integration tests didn't catch this situation, because I don't have any tests for non-probabilistic classification models (using voting instead of fully-featured probability calculation). Should add something like this.

vruusmann commented 4 years ago

@rhorrell I got this one compilation error solved, but there is more work to do before it becomes possible to transpile your test.pmml.txt sample file into 100% Java bytecode representation (eg. missing support for CompoundPredicate elements).