opensearch-project / OpenSearch

🔎 Open source distributed and RESTful search engine.
https://opensearch.org/docs/latest/opensearch/index/
Apache License 2.0
9.86k stars 1.83k forks source link

[Remote State Repositories] Introduce a new prefix for configuring remote publication repositories #16390

Open rajiv-kv opened 1 month ago

rajiv-kv commented 1 month ago

Is your feature request related to a problem? Please describe

For remote clusters, that store segment and translog in remote, the repositories are configured as node attributes with attribute prefix remote_store. This causes limitation for clusters which want to store only metadata in remote. As the repositories for cluster-state and routing-table also use the same prefix remote_store, it causes cluster-manager to incorrectly assume that publication nodes as remote nodes.

Describe the solution you'd like

We can introduce a new attribute prefix remote_publication to configure publication repositories and make the prefix configurable via static setting. Since the attributes of remote_store and remote_publication configuration don’t overlap, the validation of repository attributes during NodeJoin would be evaluated only for the corresponding node types by cluster-manager.

Related component

Cluster Manager

Describe alternatives you've considered

No response

Additional context

No response

dblock commented 2 weeks ago

[Catch All Triage - 1, 2, 3, 4]