Consider removing automatic search threadpool queue resizing

elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine

https://www.elastic.co/products/elasticsearch

Other

1.23k stars 24.85k forks source link

Consider removing automatic search threadpool queue resizing #72779

Open dakrone opened 3 years ago

dakrone commented 3 years ago

We added the ability for the search queue to be automatically resized based on Little's Law in https://github.com/elastic/elasticsearch/pull/23884, however, that work requires a desired response time to be set by a user, and thus was not very useful without knowing a great deal about your workload and topology.

With adaptive replica selection being implemented and available for quite a while now as a reasonable alternative, we should consider removing the automatic queue resizing feature in ES.

elasticmachine commented 3 years ago

Pinging @elastic/es-core-infra (Team:Core/Infra)

geekpete commented 2 years ago

This one is resolved by https://github.com/elastic/elasticsearch/pull/52280? Was deprecated in 7.x and removed in 8.x?

I also wondered if we ever tested benchmarks with both queue resizing and ARS turned on for any combined benefit? https://writequit.org/org/es/presentations/adaptive-replica-selection-presentation.html

dakrone commented 2 years ago

It was half-resolved, because we had to keep the special threadpool around internally to gather metrics required for Adaptive Replica Selection. So I would say that it was removed from a user's point of view, but not from a "tech debt" point of view.

As far as testing, I never did any testing with both enabled at the same time.