logstash-plugins / logstash-filter-grok

Grok plugin to parse unstructured (log) data into something structured.
https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html
Apache License 2.0
122 stars 97 forks source link

Thread leak on reload #132

Closed jakelandis closed 6 years ago

jakelandis commented 6 years ago

When a pipeline is reloaded the timeout enforcer thread is leaked across reloads. The thread should be stopped on reload.

Profiling with the following configuration:

input { generator { message => "hello"} } 
filter { grok { match => {"message" => "%{WORD:greeting}"}}}
output { null{} }          

Upon each reload (-r) a new thread is created, but not destroyed.

image
jakelandis commented 6 years ago

This is a bit tricky... it isn't that the thread isn't getting joined/stopped on reload (it is)...it is a bug in Logstash core that is calling initialize twice, and only 1/2 created threads are getting cleaned up. I will log a bug there soon.

For example, after 1 reload (note showing live and finished):

image

....and after 2 reloads

image
jakelandis commented 6 years ago

Root cause: https://github.com/elastic/logstash/issues/8902.

Will close this issue since there is nothing to be done from this plugin, and this a symptom of a larger problem that can affect many different plugins.