java: too many constants

[alexeyemelyanov]

Hi, tried to use TranspilerUtil.translate and JarCodeWriter to make the jar and load it at runtime:

`

    PMML pmml;
    try (InputStream resource = new FileInputStream("src/test/resources/pmml/pmml-model-with-bundle-distance.pmml")) {
        pmml = PMMLUtil.unmarshal(resource);
    }

    JCodeModel codeModel = TranspilerUtil.translate(pmml, "some.test.Klass");

    File jarFile = new File("jarfile.jar");
    try (FileOutputStream outputStream = new FileOutputStream(jarFile)) {
        Manifest manifest = ArchiverUtil.createManifest();
        CodeWriter codeWriter = new JarCodeWriter(outputStream, manifest);

        codeModel.build(codeWriter);
    }

    try {
        URL[] classpath = {
                (jarFile.toURI()).toURL()
        };

        try(URLClassLoader clazzLoader = new URLClassLoader(classpath, PMML.class.getClassLoader())){
            PMML javaPmml = PMMLUtil.load(clazzLoader);
        }
    } finally {
        jarFile.delete();
    }

`

The generated some.test.Klass contains more than 20k lines and it can'be compiled because of 'java: too many constants'.

Is there any posibility to split this huge class? or maybe another solutions?

Thanks for your help.

[vruusmann]

The generated some.test.Klass contains more than 20k lines and it can'be compiled because of 'java: too many constants'.

The above Java code snippet is 100% boilerplate, which can be used to load any size/complexity PMML model - really not helpful in troubleshooting this issue.

You better tell me what's inside your PMML file. Or if you open the generated jarfile.jar, then what's special about the main PMML$<id>.java source code file?

I've seen Javac.exe emit "code too large" errors, but I haven't seen a "too many constants" error before. The "code too large" error was covered in #4.

[vruusmann]

Just to be clear, is this a Java compiler (javac.exe) or a Java interpreter (java.exe) error?

[vruusmann]

Let's keep this issue open - I'll close it with a code change.

The problem is too many field/method declarations, which "overflows" the size limit for a Java class constant pool.

The transpiler is generating redundant utility methods currently. I believe that once they are suppressed, everything should be OK.

[alexeyemelyanov]

thank you :)

[StephanieWang]

I'm also troubled with "too many constants" problem. Is this issue fixed now? or can I try another solutions? Thank you for help.

[alexeyemelyanov]

I'm also troubled with "too many constants" problem. Is this issue fixed now? or can I try another solutions? Thank you for help.

yes, everything is fine now

[StephanieWang]

I'm also troubled with "too many constants" problem. Is this issue fixed now? or can I try another solutions? Thank you for help.

yes, everything is fine now I still have this problem. I trained lightgbm model, and transformed to pmml by jpmml-lightgbm-executable-1.3.6. The size of my pmml file is 35M. Which version of "jpmml-transpiler" could solve this issue? I tried 1.1.8 and 1.1.7, but doesn't work. Thanks for your help

[vruusmann]

I still have this problem.

@StephanieWang Please open a new issue with more details about your use case.

Ideally, can you share your model PMML with me? Simply knowing the file size of your LightGBM PMML file - 35 MB - is clearly not enough.

If I were to guess, then your model probably contains very big member decision trees (so that the Java bytecode representation of a single member decision tree doesn't fit into JVM's 65 kB limit).

Also, you could try re-training your model so that member decision trees are a bit smaller. Specifically, try configuring the max_depth parameter.

[StephanieWang]

@StephanieWang Please open a new issue with more details about your use case.

Ideally, can you share your model PMML with me? Simply knowing the file size of your LightGBM PMML file - 35 MB - is clearly not enough.

If I were to guess, then your model probably contains very big member decision trees (so that the Java bytecode representation of a single member decision tree doesn't fit into JVM's 65 kB limit).

Also, you could try re-training your model so that member decision trees are a bit smaller. Specifically, try configuring the max_depth parameter.

@vruusmann Thank you very much for your help. I open a new issue later and share the model file. Now the 'max_depth' parameter is 8, and I reduce to 6 re-training once more.