Add GraalVM native support for Jetty WebSocket

sdeleuze commented 5 months ago

Hi, based on my tests with Spring, the main broken feature with Jetty + GraalVM native image is the WebSocket support that requires to work:

GraalVM for JDK 21 to get method handle support
A missing reflection entry I am going to add on Spring side

Additional reflection entries to be added in Jetty JAR via a reflect-config.json embedded resource or in https://github.com/oracle/graalvm-reachability-metadata, see below:

[
{
"name":"org.eclipse.jetty.ee10.websocket.jakarta.common.decoders.ByteBufferDecoder",
"methods":[{"name":"<init>","parameterTypes":[] }]
},
{
"name":"org.eclipse.jetty.ee10.websocket.jakarta.common.decoders.StringDecoder",
"methods":[{"name":"<init>","parameterTypes":[] }]
},
{
"name":"org.eclipse.jetty.ee10.websocket.jakarta.common.messages.DecodedBinaryMessageSink",
"methods":[{"name":"<init>","parameterTypes":["org.eclipse.jetty.websocket.core.CoreSession","java.lang.invoke.MethodHandle","java.util.List"] }, {"name":"onWholeMessage","parameterTypes":["java.nio.ByteBuffer"] }]
},
{
"name":"org.eclipse.jetty.ee10.websocket.jakarta.common.messages.DecodedTextMessageSink",
"methods":[{"name":"<init>","parameterTypes":["org.eclipse.jetty.websocket.core.CoreSession","java.lang.invoke.MethodHandle","java.util.List"] }, {"name":"onMessage","parameterTypes":["java.lang.String"] }]
},
{
"name":"org.eclipse.jetty.websocket.common.internal.ByteBufferMessageSink",
"methods":[{"name":"<init>","parameterTypes":["org.eclipse.jetty.websocket.core.CoreSession","java.lang.invoke.MethodHandle","boolean"] }]
},
{
"name":"org.eclipse.jetty.websocket.core.messages.StringMessageSink",
"methods":[{"name":"<init>","parameterTypes":["org.eclipse.jetty.websocket.core.CoreSession","java.lang.invoke.MethodHandle","boolean"] }]
},
{
"name":"org.springframework.web.socket.adapter.jetty.JettyWebSocketHandlerAdapter",
"methods":[{"name":"onWebSocketBinary","parameterTypes":["java.nio.ByteBuffer","org.eclipse.jetty.websocket.api.Callback"] }, {"name":"onWebSocketClose","parameterTypes":["int","java.lang.String"] }, {"name":"onWebSocketError","parameterTypes":["java.lang.Throwable"] }, {"name":"onWebSocketFrame","parameterTypes":["org.eclipse.jetty.websocket.api.Frame","org.eclipse.jetty.websocket.api.Callback"] }, {"name":"onWebSocketOpen","parameterTypes":["org.eclipse.jetty.websocket.api.Session"] }, {"name":"onWebSocketText","parameterTypes":["java.lang.String"] }]
}
]

We usually recommend OSS project to start by providing support on https://github.com/oracle/graalvm-reachability-metadata side as it provides the native testing infrastructure, but maybe you prefer to add it on Jetty side (but you should then probably add native tests). Could you provide insights on how you would prefer adding such native support in Jetty (happy to setup a call to discuss that if you want)?

joakime commented 5 months ago

@olamy https://graalvm.github.io/native-build-tools/latest/maven-plugin.html

I tried a quick addition of native-maven-plugin

<groupId>org.graalvm.buildtools</groupId>
<artifactId>native-maven-plugin</artifactId>
<version>0.10.1</version>

It seems to fork another build / test of everything. Once I hacked up our build to prevent the build loops (and turn off the build-cache) that the native-maven-plugin causes, it looks like the full build will take at least twice as long now. Not sure how viable that is, we are at 1.5 hours as it is right now.

olamy commented 5 months ago

@joakime I will make some tests. But increasing so much the build time will be a pain (cache helps a lot when working on PR/branches). furthermore graalvm build will need other jvms we don't really have installed per default. What about a weekly build with -Pnative? I can add sdkman to the docker image we are using so it will be more simple to install the graalvm jdk.

olamy commented 5 months ago

@joakime I'm not quite sure to understand how/why using this plugin in the case of Jetty build? we do not really produce applications.

sdeleuze commented 5 months ago

Indeed, we usually recommend to use a dedicated profile to avoid slowing down the regular build.

@joakime native-maven-plugin is useful to run unit test in native mode. It creates a native executable that will run your test suite in native. If that's too much pain, it is also possible to just contribute related metadata and tests to https://github.com/oracle/graalvm-reachability-metadata (happy to collaborate with you on that).

olamy commented 5 months ago

Need some special setup for Jenkins as something as simple as

    <profile>
      <id>native</id>
      <build>
        <plugins>
          <plugin>
            <groupId>org.graalvm.buildtools</groupId>
            <artifactId>native-maven-plugin</artifactId>
            <extensions>true</extensions>
            <configuration>
              <!-- ... -->
            </configuration>
            <executions>
              <execution>
                <id>test-native</id>
                <goals>
                  <goal>test</goal>
                </goals>
                <phase>test</phase>
              </execution>
              <execution>
                <id>build-native</id>
                <goals>
                  <goal>compile-no-fork</goal>
                </goals>
                <phase>package</phase>
              </execution>
            </executions>
          </plugin>
        </plugins>
      </build>
    </profile>

Then

mvn clean install -Pnative

Turns into:

[ERROR] Failed to execute goal org.graalvm.buildtools:native-maven-plugin:0.10.1:compile-no-fork (build-native) on project jetty-project: Image classpath is empty. Check if your classpath configuration is correct. -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.graalvm.buildtools:native-maven-plugin:0.10.1:compile-no-fork (build-native) on project jetty-project: Image classpath is empty. Check if your classpath configuration is correct.
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:333)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.buildcache.BuildCacheMojosExecutionStrategy.execute (BuildCacheMojosExecutionStrategy.java:145)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
Caused by: org.apache.maven.plugin.MojoExecutionException: Image classpath is empty. Check if your classpath configuration is correct.
    at org.graalvm.buildtools.maven.AbstractNativeImageMojo.getClasspath (AbstractNativeImageMojo.java:394)
    at org.graalvm.buildtools.maven.AbstractNativeImageMojo.getBuildArgs (AbstractNativeImageMojo.java:192)
    at org.graalvm.buildtools.maven.AbstractNativeImageMojo.buildImage (AbstractNativeImageMojo.java:408)
    at org.graalvm.buildtools.maven.NativeCompileNoForkMojo.execute (NativeCompileNoForkMojo.java:96)

olamy commented 5 months ago

@sdeleuze I don't really understand why the native plugin doesn't skip pom type project? I have created a PR to be able to skip this see https://github.com/graalvm/native-build-tools/pull/593 There are other issues now but at least it's some progress :) merci

olamy commented 5 months ago

I made a PR https://github.com/jetty/jetty.project/pull/11712 this need to have a local build of https://github.com/graalvm/native-build-tools/pull/593

still some issues such


Failures (12):
  JUnit Jupiter:StdErrAppenderTest:testStdErrLogFormat()
    MethodSource [className = 'org.eclipse.jetty.logging.StdErrAppenderTest', methodName = 'testStdErrLogFormat', methodParameterTypes = '']
    => java.lang.Error: Cannot determine correct type for matchesSafely() method.
       org.hamcrest.internal.ReflectiveTypeFinder.findExpectedType(ReflectiveTypeFinder.java:49)
       org.hamcrest.TypeSafeMatcher.<init>(TypeSafeMatcher.java:40)
       org.hamcrest.TypeSafeMatcher.<init>(TypeSafeMatcher.java:22)
       org.hamcrest.core.SubstringMatcher.<init>(SubstringMatcher.java:15)
       org.hamcrest.core.StringContains.<init>(StringContains.java:12)
       org.hamcrest.core.StringContains.containsString(StringContains.java:31)
       org.hamcrest.Matchers.containsString(Matchers.java:472)
       org.eclipse.jetty.logging.CapturedStream.assertContains(CapturedStream.java:37)
       org.eclipse.jetty.logging.StdErrAppenderTest.testStdErrLogFormat(StdErrAppenderTest.java:42)
       java.base@21.0.3/java.lang.reflect.Method.invoke(Method.java:580)

lorban commented 4 months ago

FYI; the latest PR made by the Spring team: https://github.com/oracle/graalvm-reachability-metadata/pull/495

jetty / jetty.project

Add GraalVM native support for Jetty WebSocket #11683