Running a Helidon SE modular application with Module Layers produces mixed results

[aalmiray]

Environment Details

• Helidon Version: 2.2.1
• Helidon SE
• JDK version: 11.0.10
• OS: "mac os x", version: "10.15.7", arch: "x86_64", family: "mac"
• Docker version (if applicable):

---

Problem Description

Running a modular Helidon SE application in combination with ModuleLayer results in errors in some cases or a partly configured application in others. It appears that the Helidon codebase (while it provides full modules) does not take into account the possibility of ModuleLayer being in use by the application developer.

Steps to reproduce

1. Install the latest Layrry release. 1.1 Requires JDK 15+ 1.2 $ git clone https://github.com/moditect/layrry.git 1.3 $ cd layrry 1.4 $ mvn install 1.5 Alternartively you can download the latest release from https://github.com/moditect/layrry/releases/tag/v1.0.0.Alpha1 or install it via sdkman.
2. Create a new Helidon SE application as per the Quick Start guide https://helidon.io/docs/latest/#/se/guides/02_quickstart

3. Add a module descriptor to src/main/java

   module io.helidon.examples.quickstart.se {
   exports io.helidon.examples.quickstart.se;
   
   requires java.json;
   requires java.logging;
   requires io.helidon.webserver;
   requires io.helidon.media.jsonp;
   requires io.helidon.metrics;
   requires io.helidon.health;
   requires io.helidon.health.checks;
   }

4. Install the application on Maven Local $ mvn install
5. Run the application with all dependencies in a single layer. Use the attached layers-single.toml.txt file. (remove .txt extension) $ <larry_dir>/layrry-launcher/target/distributions/layrry-launcher-1.0.0-SNAPSHOT-dist/layrry-launcher-1.0.0-SNAPSHOT/bin/layrry --layers-config layers-single.toml

This results in the following output

Feb 12, 2021 5:27:55 PM io.helidon.common.LogConfig doConfigureLogging
INFO: Logging at initialization configured using defaults
Feb 12, 2021 5:27:55 PM io.helidon.common.HelidonFeatures features
INFO: Helidon SE 2.2.1 has no registered features
Exception in thread "features-thread" java.lang.NullPointerException
    at io.helidon.common@2.2.1/io.helidon.common.HelidonFeatures.features(HelidonFeatures.java:165)
    at io.helidon.common@2.2.1/io.helidon.common.HelidonFeatures.lambda$print$4(HelidonFeatures.java:127)
    at java.base/java.lang.Thread.run(Thread.java:834)
Feb 12, 2021 5:27:56 PM io.helidon.webserver.NettyWebServer lambda$start$7
INFO: Channel '@default' started: [id: 0x72358639, L:/0:0:0:0:0:0:0:0:60475]
WEB server is up! http://localhost:60475/greet

Notice that the application is running but some of its features failed to load.

6. Now run the application with 2 different layers. Use the attached layers.toml.txt file. (remove .txt extension) $ <larry_dir>/layrry-launcher/target/distributions/layrry-launcher-1.0.0-SNAPSHOT-dist/layrry-launcher-1.0.0-SNAPSHOT/bin/layrry --layers-config layers.toml

This results in an exception during launch:

Feb 12, 2021 5:44:06 PM io.helidon.common.LogConfig doConfigureLogging
INFO: Logging at initialization configured using defaults
Exception in thread "main" java.lang.RuntimeException: Couldn't run module main class
    at org.moditect.layrry.internal.LayersImpl.run(LayersImpl.java:139)
    at org.moditect.layrry.Layrry.launch(Layrry.java:81)
    at org.moditect.layrry.Layrry.run(Layrry.java:56)
    at org.moditect.layrry.launcher.LayrryLauncher.launch(LayrryLauncher.java:117)
    at org.moditect.layrry.launcher.LayrryLauncher.main(LayrryLauncher.java:46)
Caused by: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.moditect.layrry.internal.LayersImpl.run(LayersImpl.java:136)
    ... 4 more
Caused by: java.lang.NoClassDefFoundError: javax/json/Json
    at com.acme.helloworld@1.0-SNAPSHOT/com.acme.helloworld.GreetService.<clinit>(GreetService.java:43)
    at com.acme.helloworld@1.0-SNAPSHOT/com.acme.helloworld.Main.createRouting(Main.java:77)
    at com.acme.helloworld@1.0-SNAPSHOT/com.acme.helloworld.Main.startServer(Main.java:44)
    at com.acme.helloworld@1.0-SNAPSHOT/com.acme.helloworld.Main.main(Main.java:29)
    ... 9 more
Caused by: java.lang.ClassNotFoundException: javax.json.Json
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
    at java.base/jdk.internal.loader.Loader.loadClass(Loader.java:544)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
    ... 13 more

Layrry configures a classloader per ModuleLayer, assembling a hierarchy of layers.

[romain-grecourt]

Did you look at https://helidon.io/docs/v2/#/se/guides/37_jlink_image ? I.e mvn package -Pjlink-image

[aalmiray]

@romain-grecourt Yes, I did. It does not solve my problem as this is not a Jlink issue. The application created using the helidon-quickstart-se is not modular per se, as it does not define a module-info.java file. There are no indications that the application JAR's manifest has an Automatic-Module-Name entry either; nor there are instructions to run the application in a modular fashion (using the modulepath).

[tomas-langer]

Helidon libraries are intended for use in microservices with a single classloader (and for JPMS a single module layer). Our module info files work with JPMS (there are several integration tests validating this both for SE and MP). When running in multiple classloader or module layers, the result is undefined.

@aalmiray - can you please describe the use case that requires multiple module layers in a microservice? That would help us to better classify this issue.

Thanks!

[aalmiray]

@tomas-langer sure, I can add more detail but first just to clarify, I'm not saying Helidon does not work with the Java Platform Module System at all, just that the ModuleLayer feature may not be supported at the moment, for good reasons.

I'm currently researching how Helidon could be used to construct an application that may have part of its behavior added/removed at runtime yet remain modular. This is a feature provided by Layrry because of its usage of ModuleLayer and a single classloader per layer.

We have an example of a simple vert.x app for which additjonal http routes become a available when a plugin is added at runtime. I thought I could replicate it with Helidon SE.

[tomas-langer]

I am not sure that could work - WebServer is created and then "set in stone" - there is no way to mutate it at runtime. If you wanted to add/remove routes, you would need to stop it and create from scratch every time you want to update routing. This PR contains an example of an SE application with module-info.java: https://github.com/oracle/helidon/pull/2526/files

[aalmiray]

Understood. The example with WebServer is to demonstrate comparable behavior using plugin capabilities provided by Layrry. If Helidon's WebServer cannot have its routes mutated after the fact then that's alright, I'm not advocating for that to change. What I want is to test out Layrry's two main visible features in conjunction with Helidon:

1. Consume conflicting modules in the same application. There are times where version 1 and 2 of the same dependency (in this case Java modules) will be pulled into the dependency graph. Normally you can't have more than one "version" of the same module in the modulepath. Layrry solves this by problem by letting you define hierarchical module layers, each layer provide just enough isolation.
2. Add/remove behavior at runtime using application plugins.

If these features sound familiar is because they are (among others) provided by the OSGi spec however Layrry is much simpler than that.