Open ash-darin opened 1 year ago
This issue doesn't have a Team:<team>
label.
Hi! We just realized that we haven't looked into this issue in a while. We're sorry!
We're labeling this issue as Stale
to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1
.
Thank you for your contribution!
Describe the enhancement: Implement a "dummy" mode for the the syslog parser in filebeat, next to rfc3164 and rfc5424. This is supposed to parse Syslog Events only up to the priority level, and concatenate all lines until a new syslog priority is found at the beginnning of the line. e.g:
should result in
Being parsed as (event 1):
No further parsing is done and left to later pipelines. This explicitely not asks for this functionality to work with RFC parsing.
Describe a specific use case for the enhancement or feature: Various dumb syslog implementations of vendors (e.g. netscaler) throw events at syslog that contain newlines and ignore syslog RFCs broadly. The current syslog or TCP input can cope with neither and will for one cut these events up, resulting in syslog event spread over multiple events in elasticsearch. Additionally they will spam the filebeat log with messages (If you used SYSLOG input) that the input can not be parsed according to RFC. If the events contain newlines the only solution thus far is to have a regular syslog service read these events, dump them to a file and read that file with a multiline parser.
Most events have to be parsed later in a pipieline with logstash, if they (and they often do) ignore RFC standards.
This will avoid the error messages, cope with newlines and give a rudimentary parsing (provided syslog priority and the beginning exists)
What is the definition of done? Be able to set the fillowing in filebeat.yml:
Input syslog Events on the configured port.
If the Input contains syslog priority headers, assemble events as described above. Do not throw an error. If an "event" would exceed the "max_message_size" throw an error that the event can not be assembled.