search

jpmml / jpmml-evaluator

Java Evaluator API for PMML
GNU Affero General Public License v3.0
895 stars 255 forks source link

Compatibility with GraalVM #255

Open danilosimei opened 1 year ago

danilosimei commented 1 year ago

Hello!

I created a simple PMML service with Quarkus. Direct PMML invocation through REST endpoints didn't work because the model wasn't supported. Therefore I used JPMML implementation (https://mvnrepository.com/artifact/org.jpmml/pmml-evaluator and https://mvnrepository.com/artifact/org.jpmml/pmml-model libraries) to invoke PMML using DMN. Such setup works when running in Local Dev Mode. However there is failure when trying to generate native executable with GraalVM.

Is there any chance to make jpmml compatible with GraalVM?

vruusmann commented 1 year ago

However there is failure when trying to generate native executable with GraalVM.

I have zero working experience with GraalVM.

Can you give me a (locally reproducible-) example of your workflow, so that I could see and investigate it myself?

Is there any chance to make jpmml compatible with GraalVM?

The JPMML-Evaluator is compatible with Java SE 8. This is a pretty generous setup.

danilosimei commented 1 year ago

Hi Villu!

Error at image build:
Error: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected an instance of Random/SplittableRandom class in the image heap. Instances created during image generation have cached seed values and don't behave as expected.  To see how this object got instantiated use --trace-object-instantiation=java.util.Random. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
Detailed message:
Trace:
        at parsing com.google.common.cache.Striped64.retryUpdate(Striped64.java:184)
Call path from entry point to com.google.common.cache.Striped64.retryUpdate(long, int[], boolean):
        at com.google.common.cache.Striped64.retryUpdate(Striped64.java:182)
        at com.google.common.cache.LongAdder.add(LongAdder.java:73)
        at com.google.common.cache.AbstractCache$SimpleStatsCounter.recordLoadException(AbstractCache.java:230)
        at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2322)
        at com.google.common.cache.LocalCache$Segment$1.run(LocalCache.java:2293)
        at java.lang.Thread.run(Thread.java:829)
        at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:553)
        at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:192)
        at com.oracle.svm.core.code.IsolateEnterStub.PosixJavaThreads_pthreadStartRoutine_e1f4a8c0039f8337338252cd8734f63a79b5e3df(generated:0)

com.oracle.svm.core.util.UserError$UserException: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected an instance of Random/SplittableRandom class in the image heap. Instances created during image generation have cached seed values and don't behave as expected.  To see how this object got instantiated use --trace-object-instantiation=java.util.Random. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
Detailed message:
Trace:
        at parsing com.google.common.cache.Striped64.retryUpdate(Striped64.java:184)
Call path from entry point to com.google.common.cache.Striped64.retryUpdate(long, int[], boolean):
        at com.google.common.cache.Striped64.retryUpdate(Striped64.java:182)
        at com.google.common.cache.LongAdder.add(LongAdder.java:73)
        at com.google.common.cache.AbstractCache$SimpleStatsCounter.recordLoadException(AbstractCache.java:230)
        at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2322)
        at com.google.common.cache.LocalCache$Segment$1.run(LocalCache.java:2293)
        at java.lang.Thread.run(Thread.java:829)
        at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:553)
        at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:192)
        at com.oracle.svm.core.code.IsolateEnterStub.PosixJavaThreads_pthreadStartRoutine_e1f4a8c0039f8337338252cd8734f63a79b5e3df(generated:0)

        at com.oracle.svm.core.util.UserError.abort(UserError.java:82)
        at com.oracle.svm.hosted.FallbackFeature.reportAsFallback(FallbackFeature.java:233)
        at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:798)
        at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:582)
        at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$2(NativeImageGenerator.java:495)
        at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected an instance of Random/SplittableRandom class in the image heap. Instances created during image generation have cached seed values and don't behave as expected.  To see how this object got instantiated use --trace-object-instantiation=java.util.Random. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
Detailed message:
Trace:
        at parsing com.google.common.cache.Striped64.retryUpdate(Striped64.java:184)
Call path from entry point to com.google.common.cache.Striped64.retryUpdate(long, int[], boolean):
        at com.google.common.cache.Striped64.retryUpdate(Striped64.java:182)
        at com.google.common.cache.LongAdder.add(LongAdder.java:73)
        at com.google.common.cache.AbstractCache$SimpleStatsCounter.recordLoadException(AbstractCache.java:230)
        at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2322)
        at com.google.common.cache.LocalCache$Segment$1.run(LocalCache.java:2293)
        at java.lang.Thread.run(Thread.java:829)
        at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:553)
        at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:192)
        at com.oracle.svm.core.code.IsolateEnterStub.PosixJavaThreads_pthreadStartRoutine_e1f4a8c0039f8337338252cd8734f63a79b5e3df(generated:0)

        at com.oracle.graal.pointsto.constraints.UnsupportedFeatures.report(UnsupportedFeatures.java:126)
        at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:795)
        ... 8 more
Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected an instance of Random/SplittableRandom class in the image heap. Instances created during image generation have cached seed values and don't behave as expected.  To see how this object got instantiated use --trace-object-instantiation=java.util.Random. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
        at com.oracle.svm.hosted.image.DisallowedImageHeapObjectFeature.error(DisallowedImageHeapObjectFeature.java:163)        at com.oracle.svm.core.image.DisallowedImageHeapObjects.check(DisallowedImageHeapObjects.java:65)
        at com.oracle.svm.hosted.image.DisallowedImageHeapObjectFeature.replacer(DisallowedImageHeapObjectFeature.java:139)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.replaceObject(AnalysisUniverse.java:565)
        at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.replaceObject(AnalysisConstantReflectionProvider.java:213)
        at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.interceptValue(AnalysisConstantReflectionProvider.java:184)
        at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.readValue(AnalysisConstantReflectionProvider.java:98)
        at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.readFieldValue(AnalysisConstantReflectionProvider.java:77)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.ConstantFoldUtil$1.readValue(ConstantFoldUtil.java:51)
        at jdk.internal.vm.compiler/org.graalvm.compiler.core.common.spi.JavaConstantFieldProvider.readConstantField(JavaConstantFieldProvider.java:84)
        at com.oracle.svm.hosted.ameta.AnalysisConstantFieldProvider.readConstantField(AnalysisConstantFieldProvider.java:70)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.ConstantFoldUtil.tryConstantFold(ConstantFoldUtil.java:47)
        at com.oracle.svm.hosted.phases.ConstantFoldLoadFieldPlugin.tryConstantFold(ConstantFoldLoadFieldPlugin.java:61)        at com.oracle.svm.hosted.phases.ConstantFoldLoadFieldPlugin.handleLoadStaticField(ConstantFoldLoadFieldPlugin.java:57)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genGetStatic(BytecodeParser.java:4972)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genGetStatic(BytecodeParser.java:4939)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5442)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3451)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3258)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:1125)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:1019)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:84)
        at com.oracle.svm.hosted.phases.SharedGraphBuilderPhase.run(SharedGraphBuilderPhase.java:76)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.run(Phase.java:49)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:212)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
        at com.oracle.graal.pointsto.flow.AnalysisParsedGraph.parseBytecode(AnalysisParsedGraph.java:113)
        at com.oracle.svm.hosted.SVMHost.parseBytecode(SVMHost.java:647)
        at com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsed(AnalysisMethod.java:592)
        at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:163)
        at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:304)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.createTypeFlow(MethodTypeFlow.java:313)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureTypeFlowCreated(MethodTypeFlow.java:302)
        at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:103)
        at com.oracle.graal.pointsto.DefaultAnalysisPolicy$DefaultSpecialInvokeTypeFlow.onObservedUpdate(DefaultAnalysisPolicy.java:368)
        at com.oracle.graal.pointsto.flow.TypeFlow.notifyObservers(TypeFlow.java:470)
        at com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:542)
        at com.oracle.graal.pointsto.BigBang$2.run(BigBang.java:547)
        at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$execute$0(CompletionExecutor.java:173)
        at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
        ... 5 more
Error: Image build request failed with exit status 1

Steps to reproduce issue:

At reproducer root directory run:

mvn clean package -Pnative -Dquarkus.log.level=ALL

Java: openjdk 11.0.11 2021-04-20 OpenJDK Runtime Environment (build 11.0.11+9-Ubuntu-0ubuntu2.20.04) OpenJDK 64-Bit Server VM (build 11.0.11+9-Ubuntu-0ubuntu2.20.04, mixed mode, sharing)

GraalVM: 21.1.0

Quarkus: 2.15.1.Final

Kogito: 1.9.1.Final

PMML version: 4.4

SkLearn2PMML: 0.73.5

Scikit-learn: 0.24.2

kie-dmn-jpmml: 7.57.0.Final

JPMML-Evaluator: 1.5.15

JPMML-Model: 1.5.15

vruusmann commented 1 year ago

com.oracle.svm.core.util.UserError$UserException: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected an instance of Random/SplittableRandom class in the image heap.

If I parsed this text correctly, then the GraalVM compilation is blocked because of an unexpected java.util.Random object.

The PMML language is fully deterministic. I did a quick text search for "java.util.Random" and "random" over the JPMML-Evaluator codebase, and didn't get any matches.

Therefore, I think that some other component must be responsible for instantiating this java.util.Random object. Most likely Quarkus or Kogito.

To see how this object got instantiated use --trace-object-instantiation=java.util.Random. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=. Or you can write your own initialization methods and call them explicitly from your main entry point.

This looks like a very helpful error description.

Please re-run your experiment with --trace-object-instantiation=java.util.Random, and you should find the culprit right away. If it's any JPMML library, then I'll investigate some more.

JPMML-Evaluator: 1.5.15 JPMML-Model: 1.5.15

You should be using JPMML-Evaluator 1.6(.4) these days.

vruusmann commented 1 year ago

Anyway, GraalVM compatibility looks like an interesting research topic.

It would be interesting to AoT compile the org.jpmml.evaluator.example.EvaluationExample command-line application, and see if 1) it succeeds and 2) if makes any difference performance-wise.

danilosimei commented 1 year ago

Hi Villu,

Actually now I'm trying to build the project with the most jpmml version (which is 1.6.4) but still no success.

Now I'm getting this error when I compile to native: 

{
  "errorType": "java.util.MissingResourceException",
  "errorMessage": "Can't find bundle for base name jakarta.xml.bind.Messages, locale pt_BR"
}

Running main method

__  ____  __  _____   ___  __ ____  ______ 
--/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
-/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2022-12-28 19:28:51,934 INFO  [io.quarkus] (main) alfred-poc 1.0.0-SNAPSHOT native (powered by Quarkus 2.15.1.Final) started in 0.197s. Listening on: http://0.0.0.0:8080
2022-12-28 19:28:51,945 INFO  [io.quarkus] (main) Profile prod activated. 
2022-12-28 19:28:51,945 INFO  [io.quarkus] (main) Installed features: [amazon-lambda, amazon-s3, cdi, resteasy-reactive, resteasy-reactive-jackson, smallrye-context-propagation, vertx]
START RequestId: 7e180fa6-50f1-4f3b-909a-1fac5040d1f6 Version: $LATEST
2022-12-28 19:28:51,967 INFO  [io.neu.ser.PMMLService] (Lambda Thread (NORMAL)) Loading model
2022-12-28 19:28:54,226 ERROR [io.qua.ama.lam.run.AbstractLambdaPollLoop] (Lambda Thread (NORMAL)) Failed to run lambda (NORMAL): java.util.MissingResourceException: Can't find bundle for base name jakarta.xml.bind.Messages, locale pt_BR
at java.base@17.0.5/java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:2045)
at java.base@17.0.5/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1683)
at java.base@17.0.5/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1586)
at java.base@17.0.5/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1549)
at java.base@17.0.5/java.util.ResourceBundle.getBundle(ResourceBundle.java:858)
at jakarta.xml.bind.Messages.format(Messages.java:41)
at jakarta.xml.bind.Messages.format(Messages.java:22)
at jakarta.xml.bind.ContextFinder.newInstance(ContextFinder.java:252)
at jakarta.xml.bind.ContextFinder.newInstance(ContextFinder.java:240)
at jakarta.xml.bind.ContextFinder.find(ContextFinder.java:381)
at jakarta.xml.bind.JAXBContext.newInstance(JAXBContext.java:605)
at jakarta.xml.bind.JAXBContext.newInstance(JAXBContext.java:546)
at org.jpmml.model.JAXBUtil.getContext(JAXBUtil.java:106)
at org.jpmml.evaluator.LoadingModelEvaluatorBuilder.load(LoadingModelEvaluatorBuilder.java:151)
at org.jpmml.evaluator.LoadingModelEvaluatorBuilder.load(LoadingModelEvaluatorBuilder.java:138)
at io.neurolake.service.PMMLService.initPMML(PMMLService.java:51)
at io.neurolake.service.PMMLService_Bean.create(Unknown Source)
at io.neurolake.service.PMMLService_Bean.get(Unknown Source)
at io.neurolake.service.PMMLService_Bean.get(Unknown Source)
at io.neurolake.lambda.LambdaRequestHandler_Bean.create(Unknown Source)
at io.neurolake.lambda.LambdaRequestHandler_Bean.get(Unknown Source)
at io.neurolake.lambda.LambdaRequestHandler_Bean.get(Unknown Source)
at io.quarkus.arc.impl.ArcContainerImpl.beanInstanceHandle(ArcContainerImpl.java:476)
at io.quarkus.arc.impl.ArcContainerImpl.beanInstanceHandle(ArcContainerImpl.java:489)
at io.quarkus.arc.impl.ArcContainerImpl$2.get(ArcContainerImpl.java:279)
at io.quarkus.arc.impl.ArcContainerImpl$2.get(ArcContainerImpl.java:276)
at io.quarkus.arc.runtime.BeanContainerImpl$1.create(BeanContainerImpl.java:46)
at io.quarkus.arc.runtime.BeanContainer.beanInstance(BeanContainer.java:25)
at io.quarkus.amazon.lambda.runtime.AmazonLambdaRecorder$1.processRequest(AmazonLambdaRecorder.java:166)
at io.quarkus.amazon.lambda.runtime.AbstractLambdaPollLoop$1.run(AbstractLambdaPollLoop.java:137)
at java.base@17.0.5/java.lang.Thread.run(Thread.java:833)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:775)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:203)
END RequestId: 7e180fa6-50f1-4f3b-909a-1fac5040d1f6
REPORT RequestId: 7e180fa6-50f1-4f3b-909a-1fac5040d1f6  Duration: 2318.22 ms    Billed Duration: 2752 ms    Memory Size: 512 MB Max Memory Used: 123 MB Init Duration: 433.20 ms

I configured additional args to build the native bundle (quarkus.native.additional-build-args=-H:+IncludeAllLocales) but I'm still facing this issue above.

I have no idea how to fix this.

vruusmann commented 1 year ago

Actually now I'm trying to build the project with the most jpmml version (which is 1.6.4) but still no success

Set your system's locale to en_US (or something very similar), which should activate the default always-present message bundle.

Anyway, yuou can perform the JPMML-Evaluator upgrade once you have everything working with 1.5(.15).

Your first objective was to find out which component was creating a java.util.Random object. And my guess is that it's something else than JPMML libraries.

danilosimei commented 1 year ago

Ok Villu.

I just got back to the old version and added the trace command, here it goes:

Fatal error: com.oracle.graal.pointsto.util.AnalysisError$ParsingError: Error encountered while parsing com.google.common.cache.LongAdder.add(long)
Parsing context:
   at com.google.common.cache.LongAdder.add(LongAdder.java:68)
   at com.google.common.cache.AbstractCache$SimpleStatsCounter.recordMisses(AbstractCache.java:222)
   at com.google.common.cache.LocalCache$Segment.waitForLoadingValue(LocalCache.java:2188)
   at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2043)
   at com.google.common.cache.LocalCache.get(LocalCache.java:3966)
   at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3989)
   at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4950)
   at org.jpmml.evaluator.CacheUtil.getValue(CacheUtil.java:51)
   at org.jpmml.evaluator.ModelEvaluator.<init>(ModelEvaluator.java:157)
   at org.jpmml.evaluator.naive_bayes.NaiveBayesModelEvaluator.<init>(NaiveBayesModelEvaluator.java:105)
   at org.jpmml.evaluator.ModelEvaluatorFactory.createModelEvaluator(ModelEvaluatorFactory.java:104)
   at org.jpmml.evaluator.ModelEvaluatorFactory.newModelEvaluator(ModelEvaluatorFactory.java:79)
   at org.jpmml.evaluator.ModelEvaluatorFactory.newModelEvaluator(ModelEvaluatorFactory.java:75)
   at org.jpmml.evaluator.ModelEvaluatorFactory.newModelEvaluator(ModelEvaluatorFactory.java:61)
   at io.neurolake.service.PMMLService.initPMML(PMMLService.java:59)
   at io.neurolake.service.PMMLService_Bean.create(Unknown Source)
   at io.neurolake.service.PMMLService_Bean.get(Unknown Source)
   at io.neurolake.service.PMMLService_Bean.get(Unknown Source)
   at io.quarkus.arc.impl.ArcContainerImpl.beanInstanceHandle(ArcContainerImpl.java:476)
   at io.quarkus.arc.impl.ArcContainerImpl.beanInstanceHandle(ArcContainerImpl.java:489)
   at io.quarkus.arc.impl.BeanManagerImpl.getReference(BeanManagerImpl.java:61)
   at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:126)
   at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
   at io.quarkus.runtime.Quarkus.run(Quarkus.java:44)

        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.AnalysisError.parsingError(AnalysisError.java:153)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:104)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureFlowsGraphCreated(MethodTypeFlow.java:83)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.getOrCreateMethodFlowsGraph(MethodTypeFlow.java:65)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.typestate.DefaultVirtualInvokeTypeFlow.onObservedUpdate(DefaultVirtualInvokeTypeFlow.java:109)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:562)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.PointsToAnalysis$1.run(PointsToAnalysis.java:488)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:193)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:177)
        at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1395)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Caused by: org.graalvm.compiler.java.BytecodeParser$BytecodeParserError: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected an instance of Random/SplittableRandom class in the image heap. Instances created during image generation have cached seed values and don't behave as expected.  Object has been initialized by the com.google.common.cache.LongAddables class initializer with a trace:
        at java.util.Random.<init>(Random.java:109)
        at com.google.common.cache.Striped64.<clinit>(Striped64.java:133)
        at com.google.common.cache.LongAddables.<clinit>(LongAddables.java:35)
. Try avoiding to initialize the class that caused initialization of the object. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
        at parsing com.google.common.cache.Striped64.retryUpdate(Striped64.java:186)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.throwParserError(BytecodeParser.java:2518)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.throwParserError(SharedGraphBuilderPhase.java:110)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3393)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.handleBytecodeBlock(BytecodeParser.java:3345)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3190)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:1138)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:1030)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:97)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase.run(SharedGraphBuilderPhase.java:84)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.run(Phase.java:49)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:446)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
        at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.AnalysisParsedGraph.parseBytecode(AnalysisParsedGraph.java:135)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsed(AnalysisMethod.java:685)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.lookupEncodedGraph(InlineBeforeAnalysis.java:180)
        at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.doInline(PEGraphDecoder.java:1162)
        at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.tryInline(PEGraphDecoder.java:1145)
        at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.trySimplifyInvoke(PEGraphDecoder.java:1003)
        at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.handleInvoke(PEGraphDecoder.java:957)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.processNextNode(GraphDecoder.java:817)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.processNextNode(InlineBeforeAnalysis.java:240)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.decode(GraphDecoder.java:548)
        at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.decode(PEGraphDecoder.java:833)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysis.decodeGraph(InlineBeforeAnalysis.java:98)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:179)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:349)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:93)
        ... 13 more
Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected an instance of Random/SplittableRandom class in the image heap. Instances created during image generation have cached seed values and don't behave as expected.  Object has been initialized by the com.google.common.cache.LongAddables class initializer with a trace:
        at java.util.Random.<init>(Random.java:109)
        at com.google.common.cache.Striped64.<clinit>(Striped64.java:133)
        at com.google.common.cache.LongAddables.<clinit>(LongAddables.java:35)
. Try avoiding to initialize the class that caused initialization of the object. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.image.DisallowedImageHeapObjectFeature.error(DisallowedImageHeapObjectFeature.java:174)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.image.DisallowedImageHeapObjects.check(DisallowedImageHeapObjects.java:65)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.image.DisallowedImageHeapObjectFeature.replacer(DisallowedImageHeapObjectFeature.java:150)
        at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisUniverse.replaceObject(AnalysisUniverse.java:595)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.replaceObject(AnalysisConstantReflectionProvider.java:177)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.interceptValue(AnalysisConstantReflectionProvider.java:148)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.readValue(AnalysisConstantReflectionProvider.java:100)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.readFieldValue(AnalysisConstantReflectionProvider.java:79)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.ConstantFoldUtil$1.readValue(ConstantFoldUtil.java:55)
        at jdk.internal.vm.compiler/org.graalvm.compiler.core.common.spi.JavaConstantFieldProvider.readConstantField(JavaConstantFieldProvider.java:78)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantFieldProvider.readConstantField(AnalysisConstantFieldProvider.java:72)
        at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.ConstantFoldUtil.tryConstantFold(ConstantFoldUtil.java:51)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.ConstantFoldLoadFieldPlugin.tryConstantFold(ConstantFoldLoadFieldPlugin.java:53)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.ConstantFoldLoadFieldPlugin.handleLoadStaticField(ConstantFoldLoadFieldPlugin.java:49)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genGetStatic(BytecodeParser.java:4801)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genGetStatic(BytecodeParser.java:4772)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5282)
        at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3385)
        ... 38 more
vruusmann commented 1 year ago

This is the relevant part of the exception stack trace:

Parsing context:
   at com.google.common.cache.LongAdder.add(LongAdder.java:68)
   at com.google.common.cache.AbstractCache$SimpleStatsCounter.recordMisses(AbstractCache.java:222)
   at com.google.common.cache.LocalCache$Segment.waitForLoadingValue(LocalCache.java:2188)
   at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2043)
   at com.google.common.cache.LocalCache.get(LocalCache.java:3966)
   at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3989)
   at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4950)
   at org.jpmml.evaluator.CacheUtil.getValue(CacheUtil.java:51)
   at org.jpmml.evaluator.ModelEvaluator.<init>(ModelEvaluator.java:157)

Fundamentally, the java.util.Random object is generated by the Google Guava library (provides caching services).

Perhaps this issue can be solved by customizing the LoadingCache configuration in some way. If the configuration option fails, then it would necessitate a major-ish rewrite.

danilosimei commented 1 year ago

Yes. Well, it is not trivial at all.

I think I'll give up on trying to use graalvm.

With the newset version I could build the image but I got the locale issue (even forcing to use en_US and adding all locales while generating the image bundle).

If you have any suggestion to figure it out how to use jpmml with graalvm it wil be much appreciated!

Thanks for the help, anyway!

vruusmann commented 1 year ago

I was looking into the source code of com.google.common.cache.AbstractCache.

It appears to be the case that it's possible to choose between two com.google.common.cache.LongAddable implementations there. The default implementation (c.g.c.c.LongAdder) uses java.util.Random, but the alternative one (c.g.c.c.PureJavaLongAddable doesn't!

The logic for choosing between them is contained in com.google.common.cache.LongAddables.

There doesn't seem to be an easy/public API for overriding this logic, but I can think of a few hacks that should do.

vruusmann commented 1 year ago

There doesn't seem to be an easy/public API for overriding this logic, but I can think of a few hacks that should do.

TL;DR: Use reflection to set the com.google.common.cache.LongAddables#SUPPLIER field value to a new supplier, which always returns com.google.common.cache.PureJavaLongAddable objects.

Since the latter class has private access, it might be easier to copy&paste its contents into a new public class.

danilosimei commented 1 year ago

Ok Villu!

Thank you so much. I will try to do that. Probably it will take some time but I will try!

Again, thank you so much.