Speed Startup by 33% with the use of GraalVM Polyglot Isolates

[JaroslavTulach]

Oracle GraalVM (e.g. not the community edition, but the paid version) offers concept of isolates. Polyglot Isolates allow one to use natively compiled languages from a JVM. Such a setup would be ideal for Enso - our language could be natively compiled and our libraries could run in regular (e.g. HotSpot like) JVM.

33% Speed up

I've just finished measurements and got everything running. Simple IO.println "Hello world" application which starts in 3s or more currently (see the Startup_hello_world_startup benchmark). With the polyglot isolates the same code starts in 2s:

enso.master$ time ~/bin/graalvm-ee/bin/java \
  --module-path ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/component/  \
 --add-modules org.graalvm.polyglot Enso Demo
Saying hello...
'Hello Demo'

real    0m2,005s

E.g. we gain 33% speedup, if we repackage Enso to use isolates. Moreover we don't pay the price of using experimental Espresso as previous attempts like #8641 tried. Isolates seem to be fully compatible and fully supported by Oracle (in its commercial version of GraalVM).

Reproducer

Get the changes in PolyglotIsolates branch. Get (build 21.0.2+13-LTS-jvmci-23.1-b30) Java HotSpot(TM) 64-Bit Server VM and execute:

$ git checkout jtulach/PolyglotIsolates
$ git log | head -n5
commit 78c71ecb062edbd01e199d932385b97340e6e016
Author: Jaroslav Tulach <jaroslav.tulach@enso.org>
Date:   Sat Feb 24 11:25:15 2024 +0100

    Using GraalVM EE polyglot isolates

$ ~/bin/graalvm-ee/bin/java -version
java version "21.0.2" 2024-01-16 LTS
Java(TM) SE Runtime Environment Oracle GraalVM 21.0.2+13.1 (build 21.0.2+13-LTS-jvmci-23.1-b30)
Java HotSpot(TM) 64-Bit Server VM Oracle GraalVM 21.0.2+13.1 (build 21.0.2+13-LTS-jvmci-23.1-b30, mixed mode, sharing)

$ sbt --java-home ~/bin/graalvm-ee/ buildEngineDistribution

The regular version of Enso has just been built and can be tested. Use:

$ time ~/bin/graalvm-ee/bin/java --module-path ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/component/  --add-modules org.graalvm.polyglot Enso.java jvm RegularJVM

Saying hello...
'Hello RegularJVM'

real    0m3,469s

Then continue to building the isolates. Copy additional truffle-enterprise support and its dependencies into the built-distribution directory and build runner.so shared library:

$ cp ~/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-enterprise/23.1.2/truffle-enterprise-23.1.2.jar ./built-distribution/enso-engine-*/enso-*/component/
$ cp ~/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/nativebridge/23.1.2/nativebridge-23.1.2.jar ./built-distribution/enso-engine-*/enso-*/component/
$ sbt --java-home ~/bin/graalvm-ee/ engine-runner/buildNativeImage

with built runner.so we can use Enso with the isolates. The simplest way is:

$ time ~/bin/graalvm-ee/bin/java --module-path ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/component/  --add-modules org.graalvm.polyglot Enso.java Isolates
Saying hello...
'Hello Isolates'

real    0m3,003s

but that also includes compilation of Enso.java to Java bytecode. To avoid that compile the .java file infront of the execution. Then the time goes to 2s:

# compile first
$ ~/bin/graalvm-ee/bin/javac --module-path ./built-distribution/enso-engine-*/enso-*/component --add-modules org.graalvm.polyglot Enso.java
# execute Enso class
$ time ~/bin/graalvm-ee/bin/java --module-path ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/component/  --add-modules org.graalvm.polyglot Enso Isolates
Saying hello...
'Hello Isolates'

real    0m2,035s

[JaroslavTulach]

Few points from a discussion during morning standup on Monday Feb 26:

• licensing
  • Can we distribute binaries built on top of GraalVM EE? Yes, but whoever uses them in production needs to pay royalties
  • Are we dropping GraalVM Community support? No, it will still work in HotSpot JVM mode for Enso to continue to be purely open source project.
  • Any build system and CI work? Yes, building/testing/benchmarking with GraalVM with/without isolates.
• Is the Enso.java approach production ready?No, not at all. It is just a proof of concept:
  • Ideally we compile Enso and its launcher as native executable
  • As soon as JVM dynamic classloading is needed via polyglot java import we locate and launch a HotSpot JVM
  • There is JNI GetJavaVM method to launch a JVM in the same native process and we'd have to use it
• Can we also compile Standard.Base with Native Image?
  • Yes, but Java object/classes created by such a library won't be exchangeable with libraries that load classes dynamically
• Is the isolates based solution be faster?
  • It starts faster, but it can actually be slower during execution - jumping from an isolate to JVM (or another isolate) is going to be slower
  • Regular Graal.js, GraalPy launchers offer --jvm switch which could mitigate the problem with --jvm the startup would be slower (as no isolates would be used), but the peak performance would be as fast as now

[JaroslavTulach]

The latest OracleLabs GraalVM meetup introduced a concept of Crema project. If Crema flies, it will allow interop between libraries compiled by native image and those interpreted by Crema. Let's put the isolates work on hold for now.