New syslog codec

[suyograo]

Create a new syslog codec to add support for RFC3164 and RFC5424 (with continued ISO8601 date format support) so it can be used with any input like TCP, UDP, HTTP and so on. Plan is to deprecate the current syslog input plugin since it only supports RFC3164 and is not flexible.

The cardinality and locations of format divergence are extremely vast especially around the many networking manufacturers. Formats that diverge from the standard RFC3164 (old) and RFC5424 (newer) will require custom grok patterns to successfully parse in the Logstash ecosystem. The intent is to ubiquitously support these two popular formats standardized by centralized logging systems/forwarders like syslog-ng, rsyslog, nxlog, Kiwi syslog server, and other types of application loggers like syslog4j and Ruby syslog.

• [ ] RFC3164: https://github.com/logstash-plugins/logstash-input-syslog/issues/15
• [ ] RFC5424: https://github.com/logstash-plugins/logstash-input-syslog/issues/14
• [ ] Resolve any other important issues in the syslog input plugin: https://github.com/logstash-plugins/logstash-input-syslog/issues

Related: https://github.com/elastic/logstash/issues/1667

[suyograo]

/cc @acchen97 @tbragin

[allenmchan]

+1

[rafaelmagu]

Are there plans to support octet-framing from Rsyslog messages? We just ran into an issue where Rsyslog, when dequeueing a blocked queue, sends multiple messages to the syslog input that get interpreted as a single, ginormous one.

[valarauca]

Hello is this issue still active? My company has ran into this issue.