search

opensearch-project / OpenSearch

🔎 Open source distributed and RESTful search engine.
https://opensearch.org/docs/latest/opensearch/index/
Apache License 2.0
9.87k stars 1.84k forks source link

Making ThreadPool dynamic #3299

Open amitgalitz opened 2 years ago

amitgalitz commented 2 years ago

Is your feature request related to a problem? Please describe. Currently the Anomaly Detection plugin utilizes a custom thread pool in order to manage the amount of resources used by the AD plugin. Currently we limit the number of cores AD thread pool can use to Math.max(1, OpenSearchExecutors.allocatedProcessors(settings) / 2). However in OS 2.0-GA we will be utilizing a newer version of the Random Cut Forest algorithm that AD uses that takes advantage of multi-core processing. In order to give users the potential to fully utilize the improved speed possible with AD in OS 2.0 we want to allow users to dynamically adjust the number of cores used in the thread pool. However since these settings are static and only configured once during bootstrap of the cluster it can’t be dynamically changed.

Describe the solution you'd like I would be proposing to decouple some of the ThreadPool logic in order to make changing such setting dynamically possible. This means that a setting like the amount of processors allocated to a thread pool could be configured during runtime without completely restarting the cluster with changes to opensearch.yml

reta commented 2 years ago

@amitgalitz somewhat related to https://github.com/opensearch-project/OpenSearch/issues/476 where we made the queue size adjustable at runtime.

Bukhtawar commented 2 years ago

I think the ask is to make the thread count also configurable independent of cores

reta commented 2 years ago

I think the ask is to make the thread count also configurable independent of cores

Correct, just highlighting the related work

amitgalitz commented 2 years ago

I think the ask is to make the thread count also configurable independent of cores

Would this be required in part of making settings such as number of allocated processors assigned to the core itself dynamic?