Add support for Java Modules (Java 9+)

[thhart]

I am migrating slowly to new module system. It is painful but can not be skipped. The dependencies are handled via Maven like this there is no other dependency for BoofCV, I am developing in Idea:

        <dependency>
            <groupId>org.boofcv</groupId>
            <artifactId>boofcv-io</artifactId>
            <version>0.36.1</version>
        </dependency>

When compile I get this error:

Error:java: java.lang.module.ResolutionException: Modules boofcv.ip and boofcv.types export package boofcv.misc to module org.apache.logging.log4j.core

My module-info looks like this:

module commonsfx {
   requires javafx.graphics;
   requires javafx.swing;
   requires javafx.fxml;
   requires javafx.controls;
   requires org.apache.logging.log4j.core;
   requires org.apache.logging.log4j;
   requires com.google.common;
   requires boofcv.io;
   requires boofcv.ip;
   requires boofcv.types;
   exports com.itth.commonsfx;
   opens com.itth.commonsfx;
}

[thhart]

Additional note, the migration to module system is a mess and I am still struggling with it in general. So this report might be handled at lower priority. I have alternatives to run without module system involved still.

[lessthanoptimal]

I've not worked with modules so I'm not of much help here. Short of redesigning the project, I'm open to considering changes or creating a special module for modules.

[ex-ratt]

In principle, it should not be an issue that BoofCV does not declare modules. In this case, each jar is considered a module that exports every package that is inside the jar. What the Java module system does not allow, however, is that several modules export a package with the same name (see https://stackoverflow.com/questions/46277188/modules-a-and-b-export-package-some-package-to-module-c-in-java-9). In the given example, both boofcv-ip and boofcv-types contain a package called boofv.misc. Because of this, it is not possible to use BoofCV in a Java app that makes use of the module system (as long as you need several of BoofCV's jars in a single applicatrion).

So, if you are redesigning the project, then consider creating unique packages for each of your jars. Make a package boofcv.ip for your boofcv-ip jar, boofcv.types for boofcv-types, and so on. Then there should not be any package conflicts like that anymore and BoofCV is usable with the module system. Whether or not you add module-info.java files then is another question, but it should not be necessary.

[lanthale]

@lessthanoptimal A fatjar would be a very good solution. This can than be used easily with moditect in the java module system.

It should help if all packages names inside all jar's are unique - If you need help I can assist with my java module know-how.

[lessthanoptimal]

Creating a fatjar for everything in main/ should be straight forward. Now if a user wanted to use integration modules then that would be annoying as you would need to explicitly remove transient dependencies that match boofcv'-* Could also just have a forked project that has a different Maven Central name with a modified build script and there would be no issues then. I don't like the idea of needing to maintain that though.

[lessthanoptimal]

While a bit of a hack, I bet there's some way in gradle to grab these dependencies from Maven Central and then combine them locally.

[lanthale]

I have now created a local maven repo and installed there the fatjar created out of the core jar's for usage in my project. I am looking forward to the changes of the package names so that split packages no longer exists and adding the module-info.java for every jar.

[lessthanoptimal]

So changing package names isn't being planned right now, but a work around is under consideration. While BoofCV isn't the best at maintaining backwards compatibility, that change would absolutely break everything. In EJML there was a major refactoring and even years later a lot of people don't want to upgrade because of that.

It would also require that each sub-project have its own name space There was a recent redesign to improve build times by decoupling sub-projects. If BoofCV was compatible with Modules that change would have been major instead of being transparent to the user.

Right now the added pain factor doesn't seem to be worth the benefits since there are ways to work around it.

[lanthale]

I have an idea to get boofcv better usable on the module path: Actually you have e.g. in package org.boofcv in more than one of the lib's classes. What if you just move all classes for the package org.boofcv e.g. in the core lib. Therefore you no longer have more than one lib exporting the same packages and the developers do not need to change the code.

What do you think ?

[lanthale]

@lessthanoptimal I just want to let you know that I have shifted the usage of boofcv in Photoslide because it is not realy working with the fatjar. Sometimes maven downloads additionally the other jars for boofcv which is conflicting than with the fatjar itself. The code itself is there but with all the issues due to split packages I invest now time to other features.

[lessthanoptimal]

Thanks for the update. Out of curiosity, so you think using Modules really helps a lot for a project like yours?

On Fri, Feb 19, 2021 at 7:32 AM lanthale notifications@github.com wrote:

@lessthanoptimal https://github.com/lessthanoptimal I just want to let you know that I have shifted the usage of boofcv in Photoslide because it is not realy working with the fatjar. Sometimes maven downloads additionally the other jars for boofcv which is conflicting than with the fatjar itself. The code itself is there but with all the issues due to split packages I invest now time to other features.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lessthanoptimal/BoofCV/issues/171#issuecomment-782150087, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFUOVZTFFBQKONIMHI3HDDS7Z77PANCNFSM4PGSBJ3A .

-- "Now, now my good man, this is no time for making enemies." — Voltaire (1694-1778), on his deathbed in response to a priest asking that he renounce Satan.

[lanthale]

For me the target was to have native packages for every os and if you want to go that route it is only possible with modules. E.g the startup time of my app is cut by 50% only if I use jlink with modules and cds archives.

From a customer perspective the startup time is the key driver for a desktop application. As a benefit I can easily port the app over to iPad or Android.

You see that there are allot of advantages using modules.

[mnellemann]

I'm having lot's of trouble getting boofcv working w. a JavaFX application (worked fine w. an older Swing based Java8 app). I have tried all sort of things w. module-info.java, but keeps getting into problems w. packages being used by different modules. I anyone has a module-info.java file working w. boofcv-core, I would appreciate if I could take a look.

[lessthanoptimal]

This won't immediately help any folks using modules, but BoofCV dropped Java 8 finally and now outputs Java 11 jars. If anyone knows of a way to add module support without refactoring everything that's easier now.

[lanthale]

Just create a module-info.java in the root package for every jar. Than you are at least fine. But the problem remains that package names must be unique. That means that the lib is not usable on the module path.

[mnellemann]

Not sure how to do that?

I have tried using a gradle plugin https://github.com/jjohannes/extra-java-module-info which should help modulize non moduler libraries, but I still end of with errors like error: the unnamed module reads package boofcv.struct from both boofcv.learning and boofcv.types and error: the unnamed module reads package boofcv.alg.distort from both boofcv.geo and boofcv.ip.multiview.

[thhart]

@lessthanoptimal I tried again but got stuck on the problem reported by @mnellemann. The core problem is same packages in different jars packed which leads to an ambiguity for the module system. This is a no go for modules in general. So this really would be helpful to be avoided in general. As far as I can see this would be the last step to be able to use BoofCV within the module universe.

[thhart]

I managed to use BoofCV in a modularized system with an uber jar of BoofCV. As mentioned above the so called split package problem is the only problem which should be resolved to get BoofCV ready for it.

[lessthanoptimal]

@thhart What's needed to create an uber jar for everything in boofcv/main? Integration packages could be modularized without much repercussions.

[thhart]

Right now I am packing everything with Idea Artifacts like this and place this into the dependency tree of my Maven structures:

In principle it is a simple merge of all jars. Maven and Gradle are offering plugins for this as well but I don't know by heart how to do this. So instead of providing everything in its jars I recommend to have a BoofCV package available which includes all of these. Georegression, ddogleg might be kept outside and can be included different but ejml is the same problem of course.

[lessthanoptimal]

@thhart Can you look at this PR in DDogleg https://github.com/lessthanoptimal/ddogleg/pull/23 and tell me if I added module support correctly?

If that's good, I'll add modules to GeoRegression. This should take care of the low hanging fruit. EJML should be "easy" to make module compliant but will completely break everyone's code. Might do it anyways.

[thhart]

This was really low hanging. ;-) The module-info.java looks ok. By the way modularising something does not mean it can not be used in a non modular environment. However I see the need of change of the split packages for boofcv and ejml.

[lessthanoptimal]

Well I think I got ahead of myself. It's complaining about ejml not being a module. Trying to figure out how to use a non module project as a dependency. Any help there?

Also it looks like I have to export every single package that I want to be public and there's by design no way to do so recursively. That's a pain.

[lessthanoptimal]

https://github.com/gradlex-org/extra-java-module-info

Looks like that might allow it to build using EJML. Not sure how that affects downstream customers of DDogleg...

Also trying to find a way to auto generate the module-info.java since basically every package should be exported and needing to remember to add new packages as they are added is going to be very error prone.

[thhart]

Please note

Well I think I got ahead of myself. It's complaining about ejml not being a module. Trying to figure out how to use a non module project as a dependency. Any help there?

I you have a non modularized jar it has to be specified by the jar name as a requirement: https://openjdk.org/projects/jigsaw/spec/sotms/#automatic-modules

Also it looks like I have to export every single package that I want to be public and there's by design no way to do so recursively. That's a pain.

It is a pain yes, I was wondering also how big a module-info can become with this and yes there is no wildcard for most things. On the other hand it might be easy to traverse all package names in a script and export these. There are a couple modules helpers around but I never evaluated these. My Intellij Idea is doing a rather ok job to assist me, not perfect but so be it. I know it is a lot of struggle but with Java 17 it becomes the standard. I found some good background here: https://inside.java/2021/09/10/what-are-modules-about/