Failed to Parse Syslog Input if the APP-NAME Property is an empty string

[r0bc94]

Relevant telegraf.conf:

# # Accepts syslog messages following RFC5424 format with transports as per RFC5426, RFC5425, or RFC6587
[[inputs.syslog]]
#   ## Specify an ip or hostname with port - eg., tcp://localhost:6514, tcp://10.0.0.1:6514
#   ## Protocol, address and port to host the syslog receiver.
#   ## If no host is specified, then localhost is used.
#   ## If no port is specified, 6514 is used (RFC5425#section-4.1).
  server = "tcp://localhost:6514"
#
#   ## TLS Config
#   # tls_allowed_cacerts = ["/etc/telegraf/ca.pem"]
#   # tls_cert = "/etc/telegraf/cert.pem"
#   # tls_key = "/etc/telegraf/key.pem"
#
#   ## Period between keep alive probes.
#   ## 0 disables keep alive probes.
#   ## Defaults to the OS configuration.
#   ## Only applies to stream sockets (e.g. TCP).
#   # keep_alive_period = "5m"
#
#   ## Maximum number of concurrent connections (default = 0).
#   ## 0 means unlimited.
#   ## Only applies to stream sockets (e.g. TCP).
#   # max_connections = 1024
#
#   ## Read timeout is the maximum time allowed for reading a single message (default = 5s).
#   ## 0 means unlimited.
#   # read_timeout = "5s"
#
#   ## The framing technique with which it is expected that messages are transported (default = "octet-counting").
#   ## Whether the messages come using the octect-counting (RFC5425#section-4.3.1, RFC6587#section-3.4.1),
#   ## or the non-transparent framing technique (RFC6587#section-3.4.2).
#   ## Must be one of "octet-counting", "non-transparent".
#    framing = "non-transparent"
#
#   ## The trailer to be expected in case of non-transparent framing (default = "LF").
#   ## Must be one of "LF", or "NUL".
#   # trailer = "LF"
#
#   ## Whether to parse in best effort mode or not (default = false).
#   ## By default best effort parsing is off.
  best_effort = true
#
#   ## Character to prepend to SD-PARAMs (default = "_").
#   ## A syslog message can contain multiple parameters and multiple identifiers within structured data section.
#   ## Eg., [id1 name1="val1" name2="val2"][id2 name1="val1" nameA="valA"]
#   ## For each combination a field is created.
#   ## Its name is created concatenating identifier, sdparam_separator, and parameter name.
#   sdparam_separator = "-"

System info:

• Telegraf 1.19.0 (git: HEAD ed412d7f)
• CentOS 7.8.2003
• rsyslogd 8.24.0-52.el7

Steps to reproduce:

1. Rsyslog receives a log - message which looks like so:

   Jun 30 09:52:05 10.10.160.74 [local] 10.10.160.74 14:4e:2a:76:68:30 3904_1 RSTP-6-POSSIBLE_TOPOLOGY_CHANGE_DETECTED: chassis(1): :Rstp Detected Or Informed Possible Topology Change

2. Telegraf shows the following error message:

   E! [inputs.syslog] Error in plugin: expecting an app-name (from 1 to max 48 US-ASCII characters) or a nil value [col 44]

Expected behavior:

Telegraf should process this message and adds the parsed fields to the output (an influxdb in my case).

Actual behavior:

Only the above mentioned error message is shown.

Additional info:

Hello,

I wanted to use telegraf to process several syslog messages which are send from multiple Ciena switches. Those messages should be processed and written to an influxdb.

However, it seems that the syslog plugin has some problems processing the messages, which are send by our network switches to rsyslogd. The problem might be that the messages from our switches do not contain the APP-NAME property, however telegraf seems to complain about this.

Using the debug output of rsyslog.d, all messages of our Ciena devices looks like so:

Debug line with all properties:
FROMHOST: 'utility-server', fromhost-ip: '127.0.0.1', HOSTNAME: 'utility-server', PRI: 78,
syslogtag 'CROND[18208]:', programname: 'CROND', APP-NAME: 'CROND', PROCID: '18208', MSGID: '-',
TIMESTAMP: 'Jun 30 11:30:01', STRUCTURED-DATA: '-',
msg: '(root) CMD (/usr/lib64/sa/sa1 1 1)'
escaped msg: '(root) CMD (/usr/lib64/sa/sa1 1 1)'
inputname: imjournal rawmsg: '(root) CMD (/usr/lib64/sa/sa1 1 1)'
$!:{ "PRIORITY": "6", "_UID": "0", "_GID": "0", "_BOOT_ID": "044e5e005b824bf4b4c939527a7fa5dc", "_MACHINE_ID": "c015e7dab0ab43f9affa053432098096", "_HOSTNAME": "utility-server.pm.local", "_CAP_EFFECTIVE": "1fffffffff", "_TRANSPORT": "syslog", "_EXE": "\/usr\/bin\/bash", "SYSLOG_FACILITY": "9", "_SELINUX_CONTEXT": "system_u:system_r:crond_t:s0-s0:c0.c1023", "_AUDIT_LOGINUID": "0", "_SYSTEMD_OWNER_UID": "0", "_SYSTEMD_SLICE": "user-0.slice", "SYSLOG_IDENTIFIER": "CROND", "MESSAGE": "(root) CMD (\/usr\/lib64\/sa\/sa1 1 1)", "_COMM": "sh", "SYSLOG_PID": "18208", "_PID": "18208", "_AUDIT_SESSION": "3710", "_SYSTEMD_CGROUP": "\/user.slice\/user-0.slice\/session-3710.scope", "_SYSTEMD_SESSION": "3710", "_SYSTEMD_UNIT": "session-3710.scope", "_SOURCE_REALTIME_TIMESTAMP": "1625045401399566" }
$.:
$/:

Here is looks like the "APP-NAME" field is parsed as an empty string. The relevant parts of our rsyslog.d config looks like so:

# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514
...
$WorkDirectory /var/spool/rsyslog
$ActionQueueType LinkedList
$ActionQueueFileName srvrfwd
$ActionResumeRetryCount -1
$ActionQueueSaveOnShutdown on
$UDPServerAddress localhost
*.* @@(o)127.0.0.1:6514;RSYSLOG_SyslogProtocol23Format

It would be great if you could have a look at this.

[reimda]

Hi @r0bc94, it looks like there may be a problem in the syslog library telegraf uses. Are you still using telegraf for syslog data and are you able to help debug this further? Could you make a packet capture of the syslog data that causes the plugin error? Thanks!

[telegraf-tiger[bot]]

Hello! I am closing this issue due to inactivity. I hope you were able to resolve your problem, if not please try posting this question in our Community Slack or Community Page. Thank you!