search

opensearch-project / OpenSearch

🔎 Open source distributed and RESTful search engine.
https://opensearch.org/docs/latest/opensearch/index/
Apache License 2.0
9.79k stars 1.82k forks source link

[META] Set OpenSearch 3.0.0 baseline JDK version to JDK-21 #14011

Open reta opened 5 months ago

reta commented 5 months ago

Please describe the end goal of this project

Currently, the 3.0.0 baseline JDK version is set to JDK-11. The Apache Lucene 10 is going to require JDK-21 and it would make sense to align the OpenSearch JDK baseline requirements with that.

In order to make the migration as painless as possible, the suggestion is to start from dependent components (plugins, clients, extensions, libraries) and finish up with core, for the following reasons:

Supporting References

Please see https://github.com/opensearch-project/OpenSearch/issues/10745

Issues

Related component

Build

reta commented 5 months ago

@msfroh @andrross @dblock @getsaurabh02 fyi folks :) kicking off the campaign ...

dbwiddis commented 5 months ago

@reta Quick question on this.

In response to this campaign, I updated our compatibility from JDK 11 to JDK 21 like this:

java {
    targetCompatibility = JavaVersion.VERSION_21
    sourceCompatibility = JavaVersion.VERSION_21
}

Our normal development flow is to merge to main and backport to 2.x, which still supports earlier JDK's. I can see the potential for someone introducing code from a JDK between 12 and 20 (logically, 17) such as text blocks, switch expressions, records, and my new favorite stream operator, toList(). They'd pass muster on all our main branch CI but fail the backport and require substituting alternate code. Currently we limit this just to a very few differences to keep backporting simple.

I'm curious what the best approach here is. Can we keep targetCompatibility as 21 to address this campaign while still requiring sourceCompatibility of 11 to make sure backporting goes smoothly and doesn't diverge too much?

Or if we're keeping both, it seems this style may be more broadly applicable and recommended here. Thoughts?

tasks.withType(JavaCompile) {
    options.release.set(21)
}
andrross commented 5 months ago

@dbwiddis See some more discussion here: https://github.com/opensearch-project/custom-codecs/pull/154#discussion_r1629998769

The risk is that some new language features will be exposed by a library using JDK21 as a baseline (i.e. Lucene) and that won't work smoothly if your source compatibility is lower than that. I'm not totally sure what those risks look like in practice though.

dbwiddis commented 5 months ago

The risk is that some new language features will be exposed by a library using JDK21 as a baseline (i.e. Lucene) and that won't work smoothly if your source compatibility is lower than that. I'm not totally sure what those risks look like in practice though.

Agreed that it's not clear what the risks are. I think there are other options (e.g., ForbiddenAPIs) for enforcing lower-JDK source standards, so I'll look into those to address the risk of "dang, my backport failed". And evaluate whether it's more effort to prevent it than to just fix it when it breaks...

reta commented 5 months ago

I'm curious what the best approach here is. Can we keep targetCompatibility as 21 to address this campaign while still requiring sourceCompatibility of 11 to make sure backporting goes smoothly and doesn't diverge too much?

Thank you @dbwiddis , additionally to what @andrross said, I think the benefits of using new language features (as well as standard library) outweigh the inconvenience of manual backports (otherwise the legacy will always hold us back). Surely, there is also an option to use baseline constructs (JDK-11) without lowering the source compatibility levels.

And evaluate whether it's more effort to prevent it than to just fix it when it breaks...

I think this is sane approach, additionally we could also use something like https://github.com/openrewrite/rewrite (or similar) to "downgrade" source code during automatic backports.