sonatype-nexus-community / search-maven-org

Apache License 2.0
479 stars 141 forks source link

Treat Java module name as a first-class library property #6

Open sormuras opened 6 years ago

sormuras commented 6 years ago

Java introduced modules years ago with Java 9. By now, many library authors claimed a well-known name for their libraries. Either by specifying an Automatic-Module-Name in the MANIFEST.MF of the jar file or by an explicit module-info.class entry.

When using a library in a modular project, the end-user has to know that specific module name. As far as I know, there's no repository, search-engine or meta-crawler that exposes that information to the public.

Having an automatically updated index that makes repositories searchable for Java module names would be great to have.

tl;dr

Supersede @jodastephen 's https://github.com/jodastephen/jpms-module-names -- which offers a semi-automatic updated table of module name to group:artifact relation overview. Supersede https://github.com/sormuras/modules as well.

rgra commented 6 years ago

I second that thought.

To be able to move forward to modular projects it is neccesary that developers: 1) Know if a maven artifact is Java 9 ready with a) Having an 'Automatic-Module-Name' which makes them consumable as Java 9 Module b) Have a module-info.class which makes them fully Java 9 compatible by allowing JLinker to work which could be shown as a "Java 9" flag in two different outcomes along the artifact info in the beta UI. 2) Be able to search for a dependency by artifact name or module name. 3) Get a feeling for how many projects are already compatible, by having a list showing which dependencies are ready with respect to 1a and 1b

I get asked a lot about those points when giving Java 9 releated talks.

mvn dependency:resolve already shows the module names for all of the dependencies when running it on Java 9.

Because it is resource consuming for the community to mirror full maven central and compute the properties ourselves to get a list which is alway up to date, it would be very nice if sonatype could add some of those features to maven central.

See also the thread at: https://twitter.com/rgransberger/status/987267392878403584

sormuras commented 6 years ago

Hi @rgra

I explained this idea in more detail at https://gitter.im/sonatype-nexus-community/search-maven-org -- want to join the chat there? I will also expand the intial description with information scattered around some blog posts and other issues. Seems like a good place to collect the bits.

While we're discussing "the automated and central-ized solution", do you know some more libraries that are JPMS-ready and not listed here? Please, tell me about them by opening an issue or a PR. :)

My vision is that one day you may use JPMS module names to specify 3rd-party dependencies. A future build tool feature could also examine your module-info.java files and resolve all requires statements. Given that you somehow tell the tool which version you want. Via a module-version.properties perhaps? I already have a working proof-of-concept Gradle plugin that uses both, the hand-crafted module-maven.properties and a local module-version.properties, to achive the goal: use module names only

gunnarmorling commented 6 years ago

Hi,

My vision is that one day you may use JPMS module names to specify 3rd-party dependencies

I don't think that's a generally viable approach because there's not necessarily a 1:1 relationship between a "logical" module (i.e. a defined set of packages and classes) and a JAR artifact providing that module. E.g. in case of JCP/JarkartaEE etc. specifications, there may be multiple artifacts providing one and the same API, e.g. Hibernate's and Apache's versions of the JPA or Bean Validation APIs. They all must share the same module name (in the BV 2.0 spec, we recommend one, in future versions it'd be a mandatory part of the spec), so to be interchangeable. Ideally, there'd indeed be just a single artifact for such API which is shared across implementations, but it's not something that's going to be guaranteed.

sormuras commented 6 years ago

Ideally, there'd indeed be just a single artifact for such API which is shared across implementations, but it's not something that's going to be guaranteed.

Then please extract the API into a module and share it under a unique name (and jar). :)

Do I read your example correctly, that when I write in my ideal world: requires org.hibernate.bv I would get a jar that contains two "logical" modules, one being "javax.jpa"? When I declare instead requires org.apache.bean.validation I'd get also two "logical" module as well, also on being "javax.jpa"?

jodastephen commented 6 years ago

My view is that search.maven.org and other search engines should allow search by module name. The results of a search by module name would be a list of artifacts that match (not just one). There may be multiple in the case of EE (outlined by Gunnar) or if a project has been patched and the patch published.

It is not wrong for maven central to have multiple artifacts with the same module name, but we do need a way to find them. This means that future versions of Maven will still need groupId:artifactId, as it is not up to Maven central to define what the best mapping is from a module name to an artifact.

forax commented 6 years ago

I think that Stephen and Christian are both right :) At least, you should be able to search for a specific module by its Java name (JPMS) and you should see if a specific artifact as a module-name, a module-info.class or nothing.

but that's not enough, we start seeing artifact that are wrongly modularized, i.e. the module-info and the POM disagree on the dependencies (log4j2 had that bug 2 or 3 releases ago), so considering the POM as the source of truth, we need to verify that all the module required in the module-info are listed in the dependencies. This can be done by Maven or Nexus.

khmarbaise commented 6 years ago

@forax Hm..Maybe I misunderstand your concern related to POM relation to modules and dependencies. What about test dependencies in the pom which are usually not part of the JPMS modules (require?) ? Furthermore that might not work cause you can have multi-version jar's which are working for JDK 8, 9+...so thats a requirement which will not work...

forax commented 6 years ago

The idea is to get the required java modules from the project module-info and see if there is a corresponding module-info with the same name in one of the dependencies of the POM. If there is no project module-info, there is nothing to check :)

A required module can be declared static which means it is optional at runtime, so it should be either a normal dependencies or an optional dependencies.

Maven doesn't support to have two separate module-infos, one for the code and one for the test, so you can not validate anything for the test.

For the multi-release jar, you mean if the current project create a multi-release jar, in that case you will test it with java 8, so not testing the module-info and with java 9/10/11, in that case, you can validate the module-info.

Bascially, help people that know how to write a POM file to write a correct module-info.java.

karianna commented 6 years ago

As an FYI - we're starting some research on this at AdoptOpenJDK - See the kick off document for details.

sormuras commented 5 years ago

@jorlina, @hboutemy, Terry and me just met online and discussed next steps to get this feature into search.maven.org -- we'll continue in about 2 weeks.

In the meantime, @jodastephen's jpms-module-names project was superseded by https://github.com/sormuras/modules. That database collected already over 4.000 unique module names by parsing the published results produced by an instance of https://github.com/sandermak/modulescanner running at Sonatype's infrastructure. The sormuras/modules database is updated on a daily cadence to include the latest modular libraries published on Maven Central.

Looking forward to see Java module names, like org.objectweb.asm, pop up here... image

sormuras commented 4 years ago

The new "versions overview result page", for example for https://search.maven.org/artifact/de.sormuras.bach/de.sormuras.bach, could also show the name of the Java module provided by a JAR artifact:

image

Just between "Version" and "Updated". ;-)

sormuras commented 4 years ago

FYI: Created a similar feature request at https://youtrack.jetbrains.com/issue/PKGS-451 for Jetbrains' new Package Search product.

Here's a similar query https://package-search.jetbrains.com/package?id=de.sormuras.bach:de.sormuras.bach with a screenshot (w/o Java module API information):

image

sormuras commented 4 years ago

ping

sormuras commented 4 years ago

pong

DarthHater commented 3 years ago

Just an FYI, @jorlina and I got this in progress

DarthHater commented 3 years ago

@sormuras how would you prefer search work? That's not immediately clear to me. Displaying the info once you've found something I am figuring out, but when you go to search, how do you want to search for something on a java module name?

sormuras commented 3 years ago

how would you prefer search work?

Not sure, if this answers your question, but here are some example searches.

Search for all artifacts with this exact Java module name

Search for all artifacts that coordinates (group, artifact, module) contain this string

Basically, treat Java module names like the other coordinates as explained here: image

sormuras commented 3 years ago

See also https://github.com/sonatype-nexus-community/search-maven-org/issues/6#issuecomment-384446598 and https://github.com/sonatype-nexus-community/search-maven-org/issues/6#issuecomment-385181119.

sormuras commented 3 years ago

The Java module system was introduced last year with Java 9. [...]

Java 17 is coming!

sormuras commented 2 years ago

Is adding Java modules names to the read-only UI part of the "few other important milestones that will precede this work." mentioned here? 🤔

https://blog.sonatype.com/maven-central-and-sigstore

sormuras commented 2 years ago

The Java module system was introduced last year with Java 9. [...]

Java 17 is coming!

Java 19 is coming... 😉

Watching now https://www.youtube.com/watch?v=m9zHea6w6gs

sormuras commented 1 year ago

Java introduced modules years ago with Java 9.

Updated the initial description because Java 20 is around the corner.

emaphis commented 1 year ago

Now, Java 21 is here.

sgammon commented 6 months ago

Hello from Java 22