We are running around 150 pipelines on the same logstash and troubleshooting is becoming difficult especially because most of the pipelines are using grok and some of them are probably bad and raise the CPU to 100% full time.
I’m wondering if there is a chance to implement a property for setting a limit to backtracking, it would be great for avoiding one pipeline to take 100% CPU during whatever timeout_millis we set.
Hi,
We are running around 150 pipelines on the same logstash and troubleshooting is becoming difficult especially because most of the pipelines are using grok and some of them are probably bad and raise the CPU to 100% full time.
I’m wondering if there is a chance to implement a property for setting a limit to backtracking, it would be great for avoiding one pipeline to take 100% CPU during whatever timeout_millis we set.
this article explains it : https://api7.ai/blog/how-to-avoid-catastrophic-backtracking-completely
It’s not the key to resolve bad regex but it could minimize the effect.
I’m open to discuss it, as I read a lot of things these last few weeks and I might be wrong.
thanks for reading me. Enjoy your day