search

grafana / alloy

OpenTelemetry Collector distribution with programmable pipelines
https://grafana.com/oss/alloy
Apache License 2.0
1.45k stars 214 forks source link

Support more syslog message formats #1711

Open sboschman opened 2 months ago

sboschman commented 2 months ago

Currently loki.source.syslog only supports RFC5424 formatted messages. In #305 the issue is raised that RFC-3164 formatted messages lead to an error. PR #1556 was opened to address this with RFC-3164 support. There are other commonly used syslog formats, like CEF, by hardware vendors. As the syslog message format cannot be changed in these devices, it would be great if Alloy supported more message formats.

As adding support for every syslog message format might not be feasible, I suggest to add raw as option to the new syslog_format field, which is introduced in #1556 . This option skips any parsing and forwards the complete syslog message as is. This allows adding a 'loki_process' component in the chain and do custom parsing.

github-actions[bot] commented 1 month ago

This issue has not had any activity in the past 30 days, so the needs-attention label has been added to it. If the opened issue is a bug, check to see if a newer release fixed your issue. If it is no longer relevant, please feel free to close this issue. The needs-attention label signals to maintainers that something has fallen through the cracks. No action is needed by you; your issue will be kept open and you do not have to respond to this comment. The label will be removed the next time this job runs if there is new activity. Thank you for your contributions!

AngryMandragora commented 3 weeks ago

I am currently researching the possibility of using Loki and Alloy for collecting logs for our servers and devices, however at it's current state it's a real pain to work with. For example we have a Watchguard firewall cluster, which works perfectly with syslog-ng as a syslog server, while Alloy refuses to accept logs. Please add support for more log formats to make our lives and choices easier.

Nachtfalkeaw commented 3 weeks ago

I am currently researching the possibility of using Loki and Alloy for collecting logs for our servers and devices, however at it's current state it's a real pain to work with. For example we have a Watchguard firewall cluster, which works perfectly with syslog-ng as a syslog server, while Alloy refuses to accept logs. Please add support for more log formats to make our lives and choices easier.

You can forward these Log types to a syslog-ng/rsyslog which should convert the logs to RFC5424 which can be forwarded to alloy.

AngryMandragora commented 2 weeks ago

I am currently researching the possibility of using Loki and Alloy for collecting logs for our servers and devices, however at it's current state it's a real pain to work with. For example we have a Watchguard firewall cluster, which works perfectly with syslog-ng as a syslog server, while Alloy refuses to accept logs. Please add support for more log formats to make our lives and choices easier.

You can forward these Log types to a syslog-ng/rsyslog which should convert the logs to RFC5424 which can be forwarded to alloy.

Or use Vector to do the same, but the point would be to use only one service/program to this and not have to learn or use another.