opensearch-project / OpenSearch

🔎 Open source distributed and RESTful search engine.
https://opensearch.org/docs/latest/opensearch/index/
Apache License 2.0
9.76k stars 1.81k forks source link

[Feature Request] Introduce a cluster allocation setting to account for disk usage #15520

Open Dhruvan1217 opened 2 months ago

Dhruvan1217 commented 2 months ago

Is your feature request related to a problem? Please describe

Sometimes, In a cluster where there are indices of varied sizes, it unbalances the disks of the nodes to a great extent as currently the allocation strategy is primarily based on indices and shard counts and it doesn't account for the shard sizes while rebalancing.

Describe the solution you'd like

It would be great to have a similar setting just like we have cluster.routing.allocation.balance.index and cluster.routing.allocation.balance.shard. Elasticsearch already has cluster.routing.allocation.balance.disk_usage cluster setting which accounts for disk usage stats while performing shard balancing.

Related component

ShardManagement:Placement

Describe alternatives you've considered

N/A

Additional context

No response

andrross commented 1 month ago

[Catch All Triage - 1, 2, 3]

Thanks for filing, this looks like a reasonable feature request.