search

pires / kubernetes-elasticsearch-cluster

Elasticsearch cluster on top of Kubernetes made easy.
Apache License 2.0
1.51k stars 687 forks source link

I need to update thread_pool.bulk.queue_size: 2000 in cluster settings #216

Closed moidiqbal closed 6 years ago

moidiqbal commented 6 years ago

Im getting logstash issues while running my jobs. I need to update the key thread_pool.bulk.queue_size: 2000 manually as we couldnt set it dynamically. Thanks

[2018-08-10T18:06:09,667][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@367eb1fa on EsThreadPoolExecutor[name = es-data-0/write, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@72d2c616[Running, pool size = 1, active threads = 1, queued tasks = 200, completed tasks = 1189123]]"}) [2018-08-10T18:06:09,667][INFO ][logstash.outputs.elasticsearch] Retrying individual bulk actions that failed or were rejected by the previous bulk request. {:count=>47}

pires commented 6 years ago

You should be able to pass your own configuration by mounting it in /elasticsearch/config.

moidiqbal commented 6 years ago

any info on how to do it on running pods?

pires commented 6 years ago

You can't do it on running pods.

moidiqbal commented 6 years ago

do i need to deploy the cluster again? or is there any command that needs to be executed that restart pods with these settings?

pires commented 6 years ago

I'm sorry but I can't help if you lack understanding of the basics of Kubernetes usage, such as setting defining (mounting) paths onto Kubernetes pods.