Let's Talk: Java Modules

[gudenau]

This issue is intended for discussion related to the Java module system that was implemented in Java 9 as part of project Jigsaw.

If implemented in a way that would allow this functionality in Java 9 or greater while maintaining backwards compatibility with Java 8 should be fairly easy, but it will create issues if a backwards compatibility system of sorts is not put in place. If a developer creates a mod and only tests against Java 8 and does not specify any module metadata for Java 9+ it would have to be put into the "Unamed module".

For what I would consider a proper "backwards compatibility" needs the following:

• A way to read the module metadata (module-info.java) in mods
• A basic implementation to prevent package collisions, that violates the module system
• You don't need to prevent reflective access across "modules", but would be nice (Modules can allow reflective access with the opens keyword)
• Limited access across modules might not be possible, would be nice

Then the actual module loader implementation would require the following:

• A way to merge the conflicting Minecraft libraries, primarily the logger and Paule's Audio
• A module definition for all of the included libraries plus Minecraft itself, my toy implementations based this off of the module name in the library manifest

If this is implemented properly it would allow more flexibility with libraries and prevent reflection hacking from occurring. This prevents many issues with code accessing things that they should not, and can entirely hide internals of libraries if the need arises. It also makes it so all public fields, methods and types have to be explicitly exported. Makes it a little harder to leak info from implementations.

[RedstoneParadox]

According to the Kotlin FAQs, Kotlin can only compile to Java 6 or Java 8 bytecode, so that is also something to consider if it were to actually be a problem. No idea about Scala.

[gudenau]

It is possible to load Java 8 bytecode in a module, that is fairly trivial in fact.

[asiekierka]

We can probably use ASM to create a version-agnostic module metadata parser, and it is very likely we could even make some of its sandboxing features run - to a limited degree - on Java 8, as we control the classloader.

[gudenau]

ASM does have support for loading and generating the module meta-data.

[Pannoniae]

Are you kinda obsessed with security? Please don’t forget that we are modding a game... where you are talking about “hiding internals” or whatever. Are you sure this is the right direction? Malware often occurs for rehosted mods; and this only causes headache to everyone.

[NikkyAI]

kotlin will not be a problem: https://github.com/java9-modularity/gradle-modules-plugin recently added kotlin support, 20% according to github is kotlin, i am guessing test projects also see: https://github.com/nikolaybotevb/gradle-java9-kotlin

i have not played around with java9 and modules too much yet

and i would hope the target jvm version stays on 1.8 for the time being.. so that there are no mods requiring java9 when users only have java 8 installed

[NeumimTo]

The only reason why would you want to use java module system in a game server is to be able to use jingsaw and minimalize size of jre installed on the host machine.

Minecraft is not some complex system made out of several microservices.

"hidding internals" if you are obessed with this why dont you rather focus on creating such api that will allow to modify X without breaking Z. Same goes for reflection access.

Java modules are a powerful tool, but completly useless for mc environment

[asiekierka]

From chat, from myself:

I've worked under the assumption that modules could allow loading concurrent versions of dependencies - is that true or does the JVM still dislike two classes loaded under the same class name?

As that would be a big win if true, and possibly drop the need for shadowing

[gudenau]

Unfortunately, it would appear as though the JVM does not support discrete versions of modules being loaded at the same time; even though it can store some version info in the module metadata. [source]

This is unfortunate, but modules could still be setup to do this. It would just be super clunky or a giant hack (changing module names during loading?). Making this unsuitable for helping aid dependency conflicts.

---

Services become much easier with modules. A couple lines in the module-info.java file and a single line in a consumer of the service is all you need. Everything else is handled by the core Java libs.

Unfortunately this might not work in a Java 8 compatibility layer.

(Untested example follows, can disregard if you don't really care about a service example. :-P )

Interface example: module-info.java

module net.gudenau.servicetest.ServiceInterface{
    exports net.gudenau.servicetest.serviceinterface;
}

net/gudenau/servicetest/serviceinterface/ServiceInterface.java

package net.gudenau.servicetest.serviceinterface;

pubic interface ServiceInterface {
    void interfaceMethod();
}

Consumer example: module-info.java

module net.gudenau.servicetest.Consumer{
    requires net.gudenau.servicetest.ServiceInterface;
    uses net.gudenau.servicetest.Service;
}

net/gudenau/servicetest/consumer/Consumer.java

package net.gudenau.servicetest.consumer;

import net.gudenau.servicetest.serviceinterface.ServiceInterface;

import java.util.ServiceLoader;

public class Consumer{
    public static void main(String[] arguments){
        var loader = ServiceLoader.load(ServiceInterface.class);
        for(var service : loader){
            service.interfaceMethod();
        }
    }
}

Provider example: module-info.java

module net.gudenau.servicetest.Provider{
    requires net.gudenau.servicetest.ServiceInterface;
    provides net.gudenau.servicetest.Service
        with net.gudenau.servicetest.provider.Provider;
}

net/gudenau/servicetest/provider/ServiceInterface.java

package net.gudenau.servicetest.provider;

import net.gudenau.servicetest.serviceinterface.ServiceInterface;

public class Provider implements ServiceInterface{
    @Override
    public void interfaceMethod(){
        System.out.println("Hello from the provider!");
    }
}

[liach]

So @gudenau what is the purpose of us implementing modules? To restrict mods? I don't see any benefit here, especially given modules cannot supply different versions to different requesters.

[liach]

Hmm, an advantage of turning loader and api into java 9 modules (only if java higher than 9 is present) is that in that case mods written in java 9 or higher have the freedom to put their code in modules. otherwise, loader and api are considered as in the unnamed module, and code in named modules cannot depend on code in unnamed module.

A workaround for java 8 is to include AUTOMATIC-MODULE-NAME in jar attributes, like said in http://branchandbound.net/blog/java/2017/12/automatic-module-name/, which still exports every package of the jar and still can depend on everything else. It may be desired for loader and api to further reify and declare themselves as open modules (for mixins etc) exposing certain packages at compile time, as both only expose a selected set of packages containing api in name to their users.

[LemmaEOF]

Attempting to use Java modules in Fabric Loader currently doesn't work, which is a major issue. We should look into how to properly add support.

[liach]

Problem is the code you depend on will be in the unnamed module, like all vanilla classes. As a result, you cannot make your mod in a named module, as named modules cannot depend on unnamed modules.

[LemmaEOF]

ughhhh, really? thats garbage.

[modmuss50]

Can we not put the vanilla classes into a module?

[liach]

That will require packing all vanilla's dependencies into modules (at least automatic modules with Automatic-Module-Name jar attribute) first, and then add an Automatic-Module-Name attribute to the vanilla jar

[LemmaEOF]

we should also definitely add an Automatic-Module-Name to Fabric API, and put it in the example mod as well. Would it be possible for Loom to automatically add that?

[liach]

Should be possible.

[liach]

Hmm, now I looked at java's module api. Seems that anyone can create automatic modules than depending on java's Automatic-Module-Name scan; with java 9+'s multirelease jar, we can possibly add code that create automatic modules for mods based on fabric.mod.json, like fabric-loader.mod.modid which will then appear in stack traces.

And for modmuss's putting vanilla classes in modules: now I think that's actually possible, given the fact that we can make automatic modules from which vanilla classes are loaded.

On a side note, modules are under classloaders; each classloader has a dedicated unnamed module. I think this setup works mostly fine with the current loader, as we already load vanilla classes in knot (let's ignore launchwrapper for java 9+) and can just handle module to class relationships in the knot class loader. Since mods are loaded from jar scanning, we can just implement a module scanner that return mod jars as modules (the scanner will scan based on fabric.mod.json or the module info if it's present)

for asiekierka's library segregation, I think we can just use classloaders, right? like one classloader can load classes of library at version A for mod 1, and the other loading version B for mod 2, and mod 1 and 2 are loaded under these different class loaders. Not pro at class loading and hence I'm not sure if this approach actually works.

[teddyxlandlee]

According to the Kotlin FAQs, Kotlin can only compile to Java 6 or Java 8 bytecode, so that is also something to consider if it were to actually be a problem. No idea about Scala.

Java 16 is supported now.