Polyglot Sampler doesn't work with GraalVM for JDK 21

[JaroslavTulach]

Describe the bug

I seem to be unable to use polyglot sampler with GraalVM 23.1 for JDK 21. My application puts Truffle as well as some libraries on --module-path, VisualVM 2.1.6 or 2.1.7 can open the application and shows "Polyglot Sampler" tab. But when switching to it, no button is enabled. I see following exception on console of my application:

[error] Nov 30, 2023 8:55:44 AM org.graalvm.visualvm.sampler.truffle.stagent.TruffleJMX agentmain
[error] SEVERE: null
[error] java.lang.ClassNotFoundException: com.oracle.graalvm.locator.GraalVMLocator
[error]         at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
[error]         at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
[error]         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
[error]         at org.graalvm.visualvm.sampler.truffle.stagent.TruffleClassLoader.getClass(TruffleClassLoader.java:74)
[error]         at org.graalvm.visualvm.sampler.truffle.stagent.TruffleClassLoader.getGraalVMLocatorLoaders(TruffleClassLoader.java:91)
[error]         at org.graalvm.visualvm.sampler.truffle.stagent.TruffleClassLoader.<init>(TruffleClassLoader.java:51)
[error]         at org.graalvm.visualvm.sampler.truffle.stagent.TruffleJMX.getSamplerClassLoader(TruffleJMX.java:143)
[error]         at org.graalvm.visualvm.sampler.truffle.stagent.TruffleJMX.agentmain(TruffleJMX.java:68)
[error]         at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
[error]         at java.base/java.lang.reflect.Method.invoke(Method.java:580)
[error]         at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:560)
[error]         at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallAgentmain(InstrumentationImpl.java:582)

I guess that's the cause. Btw. looks like that GraalVMLocator is gone in GraalVM for JDK 21. The class probably no longer exists.

To Reproduce Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior

It'd be nice if polyglot sampling worked again with latest GraalVM.

Desktop (please complete the following information):

• OS: Linux
• GraalVM 23.1 for JDK 21
• Version 2.1.6

[thurka]

Please provide steps to reproduce. How do I run python or javascript?

[JaroslavTulach]

I guess this can be reproduced with any Truffle application. Originally I tried to use the sampler with Enso, but I can also reproduce it with my graal.js websocket repository:

graaljs-websocket$ git log | head -n1
commit 71f3187ad845517c0e6aa3c502ec52b85cc2b873
graaljs-websocket$ JAVA_HOME=~/bin/graalvm-community-openjdk-21+35.1/ mvn package exec:exec -DskipTests

that launches a JVM app with Graal.js. Then I connect to it via VisualVM and get the above mentioned exception. Here is even simpler application:

diff --git pom.xml pom.xml
new file mode 100644
index 0000000..94316f3
--- /dev/null
+++ pom.xml
@@ -0,0 +1,28 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <groupId>cz.xelfi.demo</groupId>
+    <artifactId>jsdemo</artifactId>
+    <version>1.0-SNAPSHOT</version>
+    <packaging>jar</packaging>
+    <properties>
+        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+        <maven.compiler.source>11</maven.compiler.source>
+        <maven.compiler.target>11</maven.compiler.target>
+        <exec.mainClass>cz.xelfi.demo.jsdemo.Jsdemo</exec.mainClass>
+        <graalvm.version>23.1.1</graalvm.version>
+    </properties>
+    <dependencies>
+        <dependency>
+            <groupId>org.graalvm.polyglot</groupId>
+            <artifactId>polyglot</artifactId>
+            <version>${graalvm.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>org.graalvm.polyglot</groupId>
+            <artifactId>js</artifactId>
+            <version>${graalvm.version}</version>
+            <type>pom</type>
+        </dependency>        
+    </dependencies>
+</project>
\ No newline at end of file
diff --git src/main/java/cz/xelfi/demo/jsdemo/Jsdemo.java src/main/java/cz/xelfi/demo/jsdemo/Jsdemo.java
new file mode 100644
index 0000000..a5f840c
--- /dev/null
+++ src/main/java/cz/xelfi/demo/jsdemo/Jsdemo.java
@@ -0,0 +1,14 @@
+package cz.xelfi.demo.jsdemo;
+
+import org.graalvm.polyglot.Context;
+
+public class Jsdemo {
+
+    public static void main(String[] args) throws Exception {
+        var ctx = Context.create("js");
+        for (var i = 0; ; i++) {
+            ctx.eval("js", "print('Hi " + i + " times');");
+            Thread.sleep(100);
+        }
+    }
+}

[thurka]

FYI: JVM Standalone from https://github.com/oracle/graaljs/releases/tag/graal-23.1.1 works fine.

[JaroslavTulach]

FYI: JVM Standalone from https://github.com/oracle/graaljs/releases/tag/graal-23.1.1 works fine.

Probably because there is modules/locator.jar in the standalone distribution. However (as pointed out by @akirathan) there is no Maven distribution of that JAR. We haven't found any and there seems to be Maven : False setting in the build script - e.g. it is not easy for us to include that JAR in the Enso release.

[thurka]

It looks to me that the above test application is missing some dependencies, since I am not able to find com.oracle.truffle.tools.profiler.CPUSampler class in any of the maven jars. Is CPUSampler available somewhere?

[JaroslavTulach]

Do you mean this CPUSampler? That one seems to be available in org.graalvm.tool:profiler-tool package.

the above test application is missing some dependencies

Do you mean that without the com.oracle.truffle.tools.profiler.CPUSampler & co. classes VisualVM cannot "polyglot sample" a JVM? I have just tried to add the org.graalvm.tool:profiler-tool:23.1.1 dependency, but I am still getting the original java.lang.ClassNotFoundException: com.oracle.graalvm.locator.GraalVMLocator error (this time with VisualVM 2.1.5)...

[thurka]

Do you mean that without the com.oracle.truffle.tools.profiler.CPUSampler & co. classes VisualVM cannot "polyglot sample" a JVM?

Right.

I am still getting the original CNFE

Yes, this needs to be fixed, but the fix for CNFE will not help if CPUSampler is not available.

[thurka]

That one seems to be available in org.graalvm.tool:profiler-tool package.

Please update the testcase above. The tescase will never work without this dependency.

[JaroslavTulach]

Here are the updated steps, Tomáši. To reproduce the steps on a Mac OS X computer copy the following "diff" into clipboard:

diff --git a/pom.xml b/pom.xml
new file mode 100644
index 0000000..71649fc
--- /dev/null
+++ b/pom.xml
@@ -0,0 +1,34 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <groupId>cz.xelfi.demo</groupId>
+    <artifactId>jsdemo</artifactId>
+    <version>1.0-SNAPSHOT</version>
+    <packaging>jar</packaging>
+    <properties>
+        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+        <maven.compiler.source>11</maven.compiler.source>
+        <maven.compiler.target>11</maven.compiler.target>
+        <exec.mainClass>cz.xelfi.demo.jsdemo.Jsdemo</exec.mainClass>
+        <graalvm.version>23.1.1</graalvm.version>
+    </properties>
+    <dependencies>
+        <dependency>
+            <groupId>org.graalvm.polyglot</groupId>
+            <artifactId>polyglot</artifactId>
+            <version>${graalvm.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>org.graalvm.polyglot</groupId>
+            <artifactId>js</artifactId>
+            <version>${graalvm.version}</version>
+            <type>pom</type>
+        </dependency>        
+        <dependency>
+            <groupId>org.graalvm.tools</groupId>
+            <artifactId>profiler-tool</artifactId>
+            <version>${graalvm.version}</version>
+            <type>jar</type>
+        </dependency>        
+    </dependencies>
+</project>
\ No newline at end of file
diff --git a/src/main/java/cz/xelfi/demo/jsdemo/Jsdemo.java b/src/main/java/cz/xelfi/demo/jsdemo/Jsdemo.java
new file mode 100644
index 0000000..a5f840c
--- /dev/null
+++ b/src/main/java/cz/xelfi/demo/jsdemo/Jsdemo.java
@@ -0,0 +1,14 @@
+package cz.xelfi.demo.jsdemo;
+
+import org.graalvm.polyglot.Context;
+
+public class Jsdemo {
+
+    public static void main(String[] args) throws Exception {
+        var ctx = Context.create("js");
+        for (var i = 0; ; i++) {
+            ctx.eval("js", "print('Hi " + i + " times');");
+            Thread.sleep(100);
+        }
+    }
+}

and in a fresh directory:

$ patch -p1
<...paste the code and Ctrl+D...>
$ JAVA_HOME=/graalvm-community-openjdk-21.0.1+12.1/Contents/Home/ mvn package -DskipTests
...
[INFO] BUILD SUCCESS

The above builds the project without any dependency problems. Then I can execute it (from NetBeans) or via CLI:

$ JAVA_HOME=/graalvm-community-openjdk-21.0.1+12.1/Contents/Home/ mvn -Dexec.args="-classpath %classpath cz.xelfi.demo.jsdemo.Jsdemo" -Dexec.executable=/graalvm-community-openjdk-21.0.1+12.1/Contents/Home/bin/java exec:exec

Once the application is running, connect to it with VisualVM and you'll see the error:

pro 06, 2023 10:56:03 DOP. org.graalvm.visualvm.sampler.truffle.stagent.TruffleJMX agentmain
SEVERE: null
java.lang.ClassNotFoundException: com.oracle.graalvm.locator.GraalVMLocator
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)

The CPUSampler class shall however be available. At least its instrument seems to be available in the process:

[thurka]

Here are the updated steps, Tomáši.

Thanks. This looks OK, however there is still some classloader mismatch. :-(

[thurka]

It looks like there is clash between classpath and module-path. Engine class is loaded by two classloaders. One is loaded by module classloader and the second is loaded by system classloader. This also affects the CPUSampler class. The code below:

            Instrument sampler = ctx.getEngine().getInstruments().get("cpusampler");
            System.out.println("Sampler "+sampler.getName());
            System.out.println("CPUSampler "+sampler.lookup(CPUSampler.class));

prints

Sampler CPU Sampler
CPUSampler null

The solution would be to use only JPMS modules, since all maven dependecies are in fact JPMS modules.

[JaroslavTulach]

There are two ways to use Truffle Unchained. --module-path and --class-path see https://medium.com/graalvm/truffle-unchained-13887b77b62c - it'd be good if there was a how to use VisualVM with Truffle Unchained provided by OracleLabs.

solution would be to use only JPMS modules

It'd be ideal to support both: --class-path as well as --module-path. However Enso has already migrated to module-path, so if there is a way to get VisualVM working in --module-path mode, with published Maven artifacts (e.g. no locator), I'll be fine with that.

Just tell me what version/commit of VisualVM I need and what modules to include. Thank you.

[thurka]

AFAICT --class-path implementation looks suspisious. I was able to modify command-line for above example to use --module-path and the truffle sampler works using attached custom build of VisualVM. BTW: How does one change pom.xml to use --module-path instead of classpath?

visualvm_GH-537.zip

[JaroslavTulach]

Hello Tomáši. I can confirm I can debug latest version of Enso with your custom VisualVM build. Thank you.

I've noticed that there are strange messages being printed on console when the VisualVM is attached:

enso.Range.fold<arg-2>(enso-dev/lib/Standard/Base/0.0.0-dev/src/Data/Range.enso:308) 2 true
enso.Standard.Base.Data.Range::Standard.Base.Data.Range.Range::fold(enso-dev/lib/Standard/Base/0.0.0-dev/src/Data/Range.enso:307) 2 true
enso.Standard.Base.Data.Vector::Standard.Base.Data.Vector.Vector::fold(enso-dev/lib/Standard/Base/0.0.0-dev/src/Data/Vector.enso:257) 2 true
enso.case_branch(enso-dev/lib/Standard/Test/0.0.0-dev/src/Bench.enso:134) 2 true
enso.Standard.Test.Bench::Standard.Test.Bench.Bench::fold(enso-dev/lib/Standard/Test/0.0.0-dev/src/Bench.enso:133) 2 true
enso.Standard.Test.Bench::Standard.Test.Bench.Bench::run_main(enso-dev/lib/Standard/Test/0.0.0-dev/src/Bench.enso:158) 2 true

is that expected in the custom build?

[thurka]

Hello Tomáši. I can confirm I can debug latest version of Enso with your custom VisualVM build.

OK. I need to do some more testing and I will fix it in master shortly.

is that expected in the custom build?

Yes, this is part of the debug output.

[thurka]

Classpath case should be fixed by just released GraalVM 23.1.2.

[JaroslavTulach]

What am I supposed to do to verify the fix, Tomáši?

• Should I use some new version of VisualVM? Where should I download it from?
• Should I upgrade Enso to GraalVM 23.1.2?

Thank you for your guidance. I tried https://github.com/enso-org/enso/pull/8883 with VisualVM 2.1.6 but I don't think the polyglot sampler is working. The exception is the same https://github.com/oracle/visualvm/issues/537#issue-2018107966 as originally reported.

[thurka]

What am I supposed to do to verify the fix, Tomáši?

You already verified it. Here: https://github.com/oracle/visualvm/issues/537#issuecomment-1855339242 . Since you are using --module-path, you don't need to update to GraalVM 23.1.2. GraalVM 23.1.2 is needed for --class-path usecase.