Opensearch Modularization

rishabhmaurya commented 2 years ago

Meta issue to track the progress of opensearch modularization.

[x] Initial investigation [ #1588]
[ ] Project base gradle configuration for module support[#1839 ]
[ ] Module naming convention [#1848 ]
[ ] Split and modularize server module [#8110]
[ ] Migrate existing modules under libs, modules directory to JPMS
[x] Migrate to lucene 9 with module support [#1109 ]
[ ] Plugins support embracing new modular architecture [#1861 ]

reta commented 2 years ago

This is a large and disruptive one, I would recommend to start with cleaning up the packages since the same ones are scattered across many modules (so called split package issue). Probably the most challenging one is org.opensearch.client which is present in server, client:rest and client:rest-high-level, at least. I am sure there are more examples like that.

rishabhmaurya commented 2 years ago

solving split packages in org.opensearch.client and modularizing will surely unblock customers using java 9 modules and the rest high level client and is a good starting point. The overall goal is to split server module and allow plugin to depend on these sub modules instead of server module. This will ensure plugins to compile and run successfully, if there are no major version upgrade in its dependencies, on opensearch major version release. Also, the packages exported by modules will be controlled, so plugins cannot depend on classes not exported explicitly. This is a big step for overall hygiene and maintainability for both opensearch and its plugins.

Taking incremental steps is the right thing but we need to ensure all breaking changes go at once in OpenSearch 2.0, so that plugins incorporate all breaking changes only once. We need to start thinking on right modules to start with and mark them as automatic module while migrating packages to them from server module, fixing split packages and cyclic dependencies. Also, when there would be these sub-modules like org.opensearch.index or org.opensearch.analyzers etc, there would be even more split packages between these newly created modules and server module. Today, there are split packages between modules under lib and server module and also for org.apache.lucence between lucene modules and server module.

reta commented 2 years ago

@rishabhmaurya I think we should make this one a [META] issue since it also drags in other projects [1]

[1] https://github.com/opensearch-project/opensearch-java/issues/108

opensearch-project / OpenSearch

Opensearch Modularization #1838