search

google / guava

Google core libraries for Java
Apache License 2.0
50.16k stars 10.9k forks source link

Add module-info.java #2970

Open hrchu opened 7 years ago

hrchu commented 7 years ago

So that projects depend on this can be published to a public artifact repository. Note that this is not breaking backward compatibility. All codes except this file can be still compiled in Java 6.

jbduncan commented 1 year ago

@bowbahdoe Wow, excellent work! I see now how sorely mistaken I was in my 2017 message when I asked if Automatic-Module-Name was sufficient. It is most certainly not. Well done discovering all this! 👏

sgammon commented 8 months ago

Has this seen any motion? We want to use jlink, but we can't because Guava is one of the last libraries on the list to ship a module-info. JDK 22 is coming this month, so JDK 8 or JDK 11 support as a new minimum would probably be okay for many users.

Guava is also one of the hardest modules to patch, replace, override, or remove entirely

This is probably a big enough and breaking enough change that it should be done from inside Google, but if you guys are open to PRs let me know.

cpovirk commented 7 months ago

It it true that this remains not a priority for us. It seems inevitable that we'll do it someday, but it's waiting behind lots of other things.

Maybe the way to go here is to identify the minimal set of changes. Like, if we are willing to depend on jdk.unsupported and java.logging for the moment (assuming that that's possible, especially for jdk.unsupported), and if we're willing to not worry about SecurityManager and finalization yet, then how much is left? There's obviously the build setup, and it looks like we'd want to migrate from the separate @J2ObjCIncompatible annotation to some package-private copies so that we can remove it as a dep. What else is there? Is the best way to find out to put the build setup in place, at least in a branch or conditionally?

There are probably PRs that we could accept, and I'd probably be willing to mess around to make sure that any patches work with, e.g. J2CL internally. But this is another case in which I don't want to give people the green light and then discover (only after people have done the work) that the changes are more disruptive than I'd expected.

bowbahdoe commented 7 months ago

@sgammon this won't help if guava is a transitive dependency, but for what its worth I am still maintaining that modularized mechanical fork. I haven't had time to update to the latest release, but i'll get to that eventually.

Another thing we can do is come up with a sanctioned moditect config - that build plugin can inject a module-info into your specific project. Not what we want long term considering how many transitive dependency graphs guava is on, but might be enough for your situation rn.

moditect, if you haven't heard of it - https://github.com/moditect/moditect

sgammon commented 7 months ago

@cpovirk I happen to be familiar with J2CL and Bazel. Let me see if I can isolate a minimal suite of changes. @bowbahdoe Thank you for this link, too, I will see if Moditect can make things simpler.

sgammon commented 7 months ago

Hey everybody,

I've gotten started on the work needed to land this. It may take awhile but I think a PR could be accomplished soon, so long as it uses some pre-packaged JARs while we wait for JPMS-enabled releases upstream. If you need Guava to support JPMS, now would be a good time to say so, so that Google knows this is worth the effort (respectfully, of course; don't hound the maintainers or anything).

Guava is number 1 in the Core Utilities category on Maven Central, and number 4 in terms of overall rank (globally) for all artifacts. Fully modular Java builds require a --modulepath and in some cases ban or frustrate the use of Automatic Modules. So, it stands to reason that fully modular Java is out of reach for most apps and libraries, if only because Guava is likely to appear somewhere in the transitive dependency graph.

JPMS is great because it offers faster compile times, better security/isolation in some ways, and so on. It is frustrating in other ways. But certainly it is not going away. Hopefully Guava supporting JPMS can make modular Java a possibility for all of us downstream. 👍

johnmanko commented 7 months ago

If you need Guava to support JPMS, now would be a good time to say so

You got my vote.

sgammon commented 7 months ago

Hello everybody!

Firstly, wow, thank you for the support on that comment! As you've probably seen from this issue, there has been a lot of motion on this from Google, and they deserve a big thank you for this (especially @cushon and @cpovirk!).

I'm commenting here now because Guava with JPMS is ready to try. It's early, but the tests all pass and there are samples building successfully.

Current state of things

Library versions

The JPMS Attic repo has all of the below libraries, i.e. everything transitively needed to fully build Guava (+ potentially your library or app) via modular Java. The repository contents are fully on GitHub for easy inspection. They don't build on GitHub yet (I'd like to do this so there is clear provenance), but they will shortly.

Until official Maven releases, these libraries have versions suffixed with -jpms, in order to avoid conflicts with Maven Central. Here's the table of artifacts available now:

Coordinate Version
com.google.errorprone:error_prone_annotations 2.25.0-jpms
com.google.guava:guava 33.0.0-jre-jpms
com.google.j2objc:j2objc-annotations 3.0.0-jpms
org.checkerframework:checker-qual 3.43.0-SNAPSHOT

Dependency Verification

Obviously, these artifacts are not yet signed with Google's GPG keys. In the meantime I have signed them with mine. This matches commit signatures on GitHub for the changes applied. Key fingerprint is:

A166AC62F6DE8C63732277731D5903AF4582BCF4
Artifact digests 
c2681472076c8da3883d5d94fb15b362acc7920e7885d53a652133450c6aa8e0  ./org/checkerframework/checker-qual/maven-metadata.xml
325419bd2c7b84445b6fafc136e9d59421b633f31022cabc4532c68f7cac8a8f  ./org/checkerframework/checker-qual/maven-metadata.xml.md5
2988c067d6d46dd5e65558d6d32172fd676cbcd6f95d2e9ac9e6c3a639f6e1f0  ./org/checkerframework/checker-qual/maven-metadata.xml.sha1
efcf78c2e21b6062e2b8113fcff6a5b7dd36ec555c82f049a7d4eed3338494c4  ./org/checkerframework/checker-qual/3.43.0-SNAPSHOT/checker-qual-3.43.0-SNAPSHOT-sources.jar
dcb92f0425cbd5fa7d9b64b46ff484d7cb9f0ba8348277501ff1f902947f276c  ./org/checkerframework/checker-qual/3.43.0-SNAPSHOT/checker-qual-3.43.0-SNAPSHOT-javadoc.jar
5ac2f9186d54139e804fc88f9cdf17034c19c499fd49ecb790bc417c3c47ad71  ./org/checkerframework/checker-qual/3.43.0-SNAPSHOT/checker-qual-3.43.0-SNAPSHOT.pom
c5ae827b30115e8eb1251c4371a9b2f2c6892be3ded43d736c86b5cdc5d66edf  ./org/checkerframework/checker-qual/3.43.0-SNAPSHOT/checker-qual-3.43.0-SNAPSHOT.module
a8224eb81224e809b741a3cecadc65be5bb3ea29964779fb120476311086f538  ./org/checkerframework/checker-qual/3.43.0-SNAPSHOT/maven-metadata-local.xml
f9a749d838626b2ba60449e267c31fb4257405ad0b2ca03b77d4dce9a7a8d700  ./org/checkerframework/checker-qual/3.43.0-SNAPSHOT/checker-qual-3.43.0-SNAPSHOT.jar
760f8a465b22877547761227fa6fe9e77b68488841beb76283285b8775f862ed  ./com/google/guava/guava/maven-metadata.xml
8b267cd72e328f211c17a38890516ed2203ae877d905e9cdd15a95c632d818a1  ./com/google/guava/guava/33.0.0-jre-jpms/guava-33.0.0-jre-jpms.jar.md5
920e137661a12a7ccb32933568a34e6a05be01d0de87c32ff73749b9f8c506d4  ./com/google/guava/guava/33.0.0-jre-jpms/guava-33.0.0-jre-jpms.pom.md5
84626ef4bfbef95e2f9d7d0983a0e8829376697a7d07d6c3de328d445f9b88b8  ./com/google/guava/guava/33.0.0-jre-jpms/guava-33.0.0-jre-jpms.jar.sha1
3ed27fa5bdc2589d1d1227c4c99c1db1c0117c7f30bd6acebddb1ddd58f960fd  ./com/google/guava/guava/33.0.0-jre-jpms/guava-33.0.0-jre-jpms.jar
a7f41c1cfea3fd7eb0fc305a155b7d4931fc692a5c0a077d202d92afed14d82c  ./com/google/guava/guava/33.0.0-jre-jpms/guava-33.0.0-jre-jpms.pom.sha1
9fe95275923730b89704c2fd6d85fd7a8880e688716a558aee61f8f5fb60895f  ./com/google/guava/guava/33.0.0-jre-jpms/guava-33.0.0-jre-jpms.pom
3c0b16b1c401730cd57519f4a9cdb459fab977634eb05396dec238308e06189d  ./com/google/guava/guava/maven-metadata.xml.md5
4c62d866a06ec05f7d7cc1299534229863f9e505afd7e78bd80fa9bdb0482e1e  ./com/google/guava/guava/maven-metadata.xml.sha1
511b91d2a05c18a0384491fb27ca6b1b3354e8dd7a5fe42a70d2f8058fb9e4b3  ./com/google/errorprone/error_prone_annotations/maven-metadata.xml
b9f01dc5b94ad3b25c84f82c53949c8e035d54f88fd839130f45507a53e5f6c4  ./com/google/errorprone/error_prone_annotations/maven-metadata.xml.md5
604f844f85afee7607b96a639e4b404f43a8431c964c87ca34c1dfed346fb2a2  ./com/google/errorprone/error_prone_annotations/maven-metadata.xml.sha1
081ed76568a21a1c23a2f70b793cad4003550ee8ece0f89974b7217af44dbf27  ./com/google/errorprone/error_prone_annotations/2.25.0-jpms/error_prone_annotations-2.25.0-jpms.jar.md5
3b0060a8dec47f9a0281d381ef9431b3b2409ab211a93e1e79208f4239feb6ce  ./com/google/errorprone/error_prone_annotations/2.25.0-jpms/error_prone_annotations-2.25.0-jpms.pom.md5
89a49c9861b01fecc123f469c856b4b2e166eecf7a1fdabdbc9c30f06e33a6d4  ./com/google/errorprone/error_prone_annotations/2.25.0-jpms/error_prone_annotations-2.25.0-jpms.jar.sha1
8745e4577e4c23611b021641bbded603bfbac9edc322bec41623f909aefa88a1  ./com/google/errorprone/error_prone_annotations/2.25.0-jpms/error_prone_annotations-2.25.0-jpms.pom
65fbc0149dca836a304bfb74b4c7d113d278ec8e4f83c9687cbeef130836bfa1  ./com/google/errorprone/error_prone_annotations/2.25.0-jpms/error_prone_annotations-2.25.0-jpms.jar
bb48f614d531d53bde372b0d3f6e6bf8b8ec1787b3388f24db64600192b4b923  ./com/google/errorprone/error_prone_annotations/2.25.0-jpms/error_prone_annotations-2.25.0-jpms.pom.sha1
a813595eb6988d03fca84942769cddd7fc9ec324ba298240e5561c6abc31d807  ./com/google/j2objc/j2objc-annotations/maven-metadata.xml
caf413a656b17e817fb7a6750fa34423a0f9c59a8b630ba3ea065fdb6732e8ba  ./com/google/j2objc/j2objc-annotations/maven-metadata.xml.md5
07c18b57336a6c1e06c4c6810bd92442f5ceafbc7fdc52a964fd352d74a2a3a9  ./com/google/j2objc/j2objc-annotations/maven-metadata.xml.sha1
c9528d7c15ae180a24ff248ed189fd1e7fd12ed84ea7a8a81ef8d23dd1a1919d  ./com/google/j2objc/j2objc-annotations/3.0.0-jpms/j2objc-annotations-3.0.0-jpms.pom.md5
e6f5e5c4471eedba428638b51475c54b789e9c695fd3905a11c5816107bc5fdb  ./com/google/j2objc/j2objc-annotations/3.0.0-jpms/j2objc-annotations-3.0.0-jpms.jar.sha1
29994c04ca63f8f81d26ae36a1f90ff59454238dead14e4d33cc80af8fcc488f  ./com/google/j2objc/j2objc-annotations/3.0.0-jpms/j2objc-annotations-3.0.0-jpms.jar.md5
cd20c532f794f30046a089354727b215788a7077bc3ae34bc6cab31d4ca7de81  ./com/google/j2objc/j2objc-annotations/3.0.0-jpms/j2objc-annotations-3.0.0-jpms.pom
a9575f66820a1d03d78da53e2e60a290eaadbe2543837d09e6a28be18b17847e  ./com/google/j2objc/j2objc-annotations/3.0.0-jpms/j2objc-annotations-3.0.0-jpms.jar
419b97e5e51386e6e301015262795ad212d0d6f698e7a99db35dc5029ddd6433  ./com/google/j2objc/j2objc-annotations/3.0.0-jpms/j2objc-annotations-3.0.0-jpms.pom.sha1

Trying it out

Here are some inlined instructions for using the JPMS Attic repo in your build. If something isn't covered here that you need, check the GitHub repo, file an issue, or let me know here with a tag:

Maven

There is a Maven sample used in integration tests

  <repositories>
    <repository>
      <id>jpms-attic</id>
      <name>JPMS Attic</name>
      <url>https://jpms.pkg.st/repository</url>
    </repository>
  </repositories>

Gradle

There is a Gradle sample used in integration tests

Groovy
repositories {
    maven {
        url "https://jpms.pkg.st/repository"
    }
}
Kotlin
repositories {
    maven {
        url = uri("https://jpms.pkg.st/repository")
    }
}

Other notes

Would you rather use a Maven BOM, or a Gradle Version Catalog? Do you need to constrain versions to these JPMS libraries transitively, using a Gradle Platform? All of these ancillary artifacts are also available from the JPMS attic repo.

ben-manes commented 7 months ago

There are probably PRs that we could accept, and I'd probably be willing to mess around to make sure that any patches work ... internally. But this is another case in which I don't want to give people the green light and then discover (only after people have done the work) that the changes are more disruptive than I'd expected.

@sgammon Please be aware of this comment and Guava's general stance on contributions. A lot of necessary work, but also racing well ahead of the maintainers and their priorities. It is often best to think of Google OSS as shared with the community, rather than driven by it, so as to realize it closely follows Google's internal needs rather than external ones. Your PRs are interesting and good work, but also quite large with Caliper to JMH migration, Java clean up, and lots of CI & build additions. Yet, most Googlers are only intimately familiar with Google's internal stack (Bazel, TAP) which also builds Guava for its usage. Therefore much of your work has to replicated into their internal stack and given a lack of a Google champion, that seems like a high bar.

I really like your ideas and direction but as a former Googler its hard for me to get bug fixes in from the outside. I'd recommend after proving it all out to split your work down into into small PRs with clear objectives, which could then be phased in incrementally as the Guava team has time. And certainly don't get demotivated by what I say, just be realistic that your work might be a PoC that guides them to support once that becomes a necessity.

sgammon commented 7 months ago

Hey @ben-manes,

Firstly, big fan of your work... Gradle Versions, Caffeine, so many libraries. I built an entire company (partly) on Caffeine. Shameless plug for Buildless, which soon will be fully free forever for open source.

I hope you will allow me to respond out-of-order here.

Your PRs are interesting and good work [...] I really like your ideas and direction

Want to say ahead of the rest of this that I will frame this in my mind. You're a giant and that's very kind. Thank you!

It is often best to think of Google OSS as shared with the community, rather than driven by it, so as to realize it closely follows Google's internal needs rather than external ones

You could not be more correct and that is a very healthy way to think about it.

Please be aware of this comment and Guava's general stance on contributions. A lot of necessary work, but also racing well ahead of the maintainers and their priorities

I am aware, and I know that JPMS is arguably a new feature, even though it maybe shouldn't be considered as such. I could argue that we're not that ahead of the maintainers here, given that the Guava team has had this under consideration for some time (speaking about just JPMS here for a moment). Your point stands, though, regardless of those arguments, that JPMS is not a priority inside Google, at least for OSS, and @cpovirk has been clear and honest about that, to his great credit.

Luckily, I've contributed to libraries at Google before (though nothing as widely used as Guava), so this is not a surprise at all and I totally understand that Googlers straddle several (sometimes diverging) priorities. The first few of these PRs are already merged on Error Prone, J2ObJC, and on Guava.

but also quite large with Caliper to JMH migration, Java clean up, and lots of CI & build additions

This was and remains my fear as well, and so I have split off the JDK 21 and CI stuff entirely. Each stands on its own, and the CI stuff is entirely optional. If the Guava team chooses not to merge these, no worries, we can rebase the JPMS pull cleanly and move on.

That said, I think you're right about Caliper vs. JMH. During that work I actually did figure out that the benchmarks were running from Maven, though probably not... reporting anywhere or doing anything. In any case, they should probably be left alone so that internal Google tooling stays working. At the very least this should be another split point in the PR.

The bench changes, thankfully, were made very late in the PR and stand alone, so those can be split away cleanly.

And certainly don't get demotivated by what I say, just be realistic that your work might be a PoC that guides them to support once that becomes a necessity

This is excellent feedback and I deeply appreciate you taking the time to share it. I don't find it demotivating at all. In fact, I am only further motivated because the feedback that has poured in so far has been great, including yours. Some of it positive, some of it not, but all really really useful and helpful and engaged.

I am consuming these artifacts myself for my own projects from the JPMS attic repo, and they are working fine after a few tweaks, so my code is not blocked by this, and others have an option to try it out early, too. I think, all of the above considered, the Guava project can take their time and we (the group of people who care about this, in this issue) can test, refine, and eventually get a very clear plan going that minimizes breakage downstream.

Anyway, thank you for sharing your thoughts. Honesty is an expensive gift.

jbduncan commented 7 months ago

@sgammon I'm in awe of your realistic and professional yet persistent approach to getting JPMS support and a better CI setup into Guava, as I'm acutely aware how Guava is more of a "throw over the fence" project than a community project, so well done and I wish you all the best moving things forward! 👍

sgammon commented 7 months ago

@jbduncan Thank you! That's very kind of you, but I'm standing on the shoulders of giants here. This issue was filed nearly 10 years ago. We should make custom jackets.

Update: Taking orders

Front Back
bowbahdoe commented 7 months ago

@sgammon I'll take one, but maybe just GUAVA 2970 on the front. I don't actually want anyone to ask me questions

cpovirk commented 3 months ago

The toolchain setup of https://github.com/google/guava/pull/7333 may lay the groundwork for continuing to run tests under JDK 8 even after we requires JDK 9 to build (at least if you want module-info). It may well be duplicative of existing work in #7094, which I still need to get back to :(

sgammon commented 3 months ago

@cpovirk Thank you for the ping; no worries, I'm happy to rebase or construct it again from scratch if needed.