search

logstash-plugins / logstash-filter-jdbc_streaming

A Logstash filter that can enrich events with data from a database
Apache License 2.0
12 stars 23 forks source link

Unhandled exception when connection fails #19

Open synFK opened 5 years ago

synFK commented 5 years ago

logstash (5.6.14 and 6.5.4) _logstash-filter-jdbcstreaming (1.0.4)

Hi there,

I get an error when connecting to a dababase server that refuses the connection. Unfortunately this error kills the whole pipeline and no further events are processed. In my understanding the tags from tag_on_failure should be applied in such a error case.

synFK commented 5 years ago

Okay, this is only true for the initial registering phase. Shutting the server down while being in the filtering phase leads to a handled exception which results in the desired tags. But: it also closes/resets the connection and does not open it on the next event, even if the server has started again.

synFK commented 5 years ago

So, if I get it right the only way to reestablish the connection is to restart Logstash. That is kind of inacceptable in production mode, isn't it? I can not imagine, that we are the only ones with this issue. Or are the others like "Our database server simply never goes down."? Whatever, I suggest a new mechanism, that handles database connection loss. Could be the options connection_retry_attempts and connection_retry_attempts_wait_time grabbed from Jdbc input plugin. Please comment.

CedricP-AM commented 5 years ago

+1 I have got the exact same problem : the pipeline definitely stops processing events when the database server stops and restarts for nightly maintenance. Every morning I need to restart logstash.

synFK commented 5 years ago

@CedricP-AM there is a PR addressing this issue (https://github.com/logstash-plugins/logstash-filter-jdbc_streaming/pull/22). Also, you might have a look a this fork: https://github.com/synFK/logstash-filter-jdbc_streaming/tree/feature/retry_mechanism.