The fact is: http calls are way more slower than filter processing.
Http calls don't require CPU load. It is waiting time for the worker.
And so, http calls dramatically slow down pipeline workers throughput.
Pipeline workers count should be tied to CPU load, to match filter processing needs (BTW, workers count default value is CPU count).
So it would be great that http calls are really asynchronous, with detached threads.
Detached threads count should be equal to pool_max setting.
Thanks to this, pipeline workers wouldn't be impacted by http calls.
Currently, the http call is tied to current worker thread. Especially, because of this line: https://github.com/logstash-plugins/logstash-output-http/blob/2cca9dc4acea7dadd203a6b0645bac0d825888d1/lib/logstash/outputs/http.rb#L239
The fact is: http calls are way more slower than filter processing. Http calls don't require CPU load. It is waiting time for the worker. And so, http calls dramatically slow down pipeline workers throughput. Pipeline workers count should be tied to CPU load, to match filter processing needs (BTW, workers count default value is CPU count).
So it would be great that http calls are really asynchronous, with detached threads. Detached threads count should be equal to
pool_maxsetting. Thanks to this, pipeline workers wouldn't be impacted by http calls.