search

logstash-plugins / logstash-output-gelf

Apache License 2.0
15 stars 18 forks source link

Gelf output with multiple hosts for "silent" fail over #4

Open jordansissel opened 9 years ago

jordansissel commented 9 years ago

(This issue was originally filed by @rlanore at https://github.com/elastic/logstash/issues/2460)

Given a cluster of gelf nodes, it would be desirable that LS silently fail over in case a gelf node, pointed by LS, dies.

gergelybosze commented 7 years ago

Hi, I see this issue is still open, it means this logstash gelf output failover feature is not implemented yet? Is there a plan for it? Do we have to think about another loadbalancer solution? Thanks!

pastukhov commented 5 years ago

Any news?

afreiberger commented 2 years ago

I also desire this feature.

While investigating the implementation, I've found the following:

The GELF output plugin calls the GELF::Notifier initializer with a single host and port argument.

https://github.com/logstash-plugins/logstash-output-gelf/blob/main/lib/logstash/outputs/gelf.rb#L86

There is a limitation of the GELF::Notifier module that assumes a single back-end host both in the initiator method as well as the call to the transport layer in the create_sender method. https://github.com/graylog-labs/gelf-rb/blob/master/lib/gelf/notifier.rb#L25 https://github.com/graylog-labs/gelf-rb/blob/master/lib/gelf/notifier.rb#L137

Looking at the transport modules for TCP, TCP-TLS, and UDP, they all implement an idea of rotation through multiple sockets until a message is sent validly if multiple addresses are provided, but the create_sender method in the GELF::Notifier limits this to a single host and port at: https://github.com/graylog-labs/gelf-rb/blob/master/lib/gelf/notifier.rb#L138.

To resolve this, the upstream GELF::Notifier library would need an update to handle multiple hosts during instantiation and then this logstash plugin would need to have an expanded definition in it's schema and routine that instantiates the GELF::Notifier.