BSD Syslog from mikrotik not parsed

luismanson commented 4 years ago

Hello, I need to receive logs from a Mikrotik router but found it does not work 100%.

There in an option in Mikrotik settings named 'bsd-syslog' which in their docs says "whether to use bsd-syslog as defined in RFC 3164".

Expected Behavior

Receive 'bsd style' syslog messages without tweaking any further. That way Graylog could process the full message.

Current Behavior

Messages are received only without 'bsd-syslog' setting. 'Allow overriding date' has to be enabled too, in Graylog's input.

Received messages don't have their information extracted automatically.

Context

I've linked a pcap from the graylog server which show both kind of messages. The first two, are sent with 'bsd-syslog' and NOT received into graylog, last three are received ok but messages are saved into the full_message field only, nothing is extracted.

full_message
    system,info,account mk_MDZi: user example logged out from 200.12.34.56 via winbox

Pcap: https://filebin.net/ardm8t32t26fevhy

Im learning about graylog, but without this, more work has to be done to process incoming messages.

Your Environment

Graylog Version: 3.2.4+a407287
Elasticsearch Version: 6.8.8
MongoDB Version: 3
Operating System: Ubuntu amd64 / Docker

radykal-com commented 4 years ago

Hi, can you share the commands you used to configure the mikrotik logging? I have a pair mikrotik routers at home and I can try to find what's going on

luismanson commented 4 years ago

Hello, The device where I found the issue is running outdated software, that might be the cause for this, but I can not upgrade now. I'm sorry i did not see that before.

Still did some tests in an up-to-date router and it works with some differences between settings:

/system logging action
add bsd-syslog=yes name=GraylogA remote=192.168.77.6 remote-port=1514 src-address=192.168.77.1 syslog-facility=syslog target=remote

Graylog received messages

facility has the same value as the 'syslog-facility' setting in the router. _fullmessage includes local time at the start of the message <46>May 11 16:21:35 rb... and does not include facilities. level is ok source shows '/system identity' value Message screenshot: https://i.imgur.com/splLos2.png

/system logging action
add name=GraylogB remote=192.168.77.6 remote-port=1514 src-address=192.168.77.1 syslog-facility=syslog target=remote

Graylog received messages

facility is "Unknown" _fullmessage has facilities (script, debug), does not include time level is "-1" source is the device IP address (not device name) Message screenshot: https://i.imgur.com/phiR02K.png

Thanks

lcosmin commented 2 years ago

Hi,

Here are some tcpdump packets from an up-to-date mikrotik device, using the "BSD Syslog" option.

I don't see the messages in Graylog, although the input receives traffic (i see the metrics move up and down). I'm using a Syslog UDP input configured on port 5140, graylog is 4.2.6-1 on an Ubuntu 21.10

13:32:23.048348 IP (tos 0x0, ttl 64, id 55670, offset 0, flags [DF], proto UDP (17), length 80)
    172.19.3.1.59825 > 172.19.3.50.5140: UDP, length 52
        0x0000:  4500 0050 d976 4000 4011 02cd ac13 0301  E..P.v@.@.......
        0x0010:  ac13 0332 e9b1 1414 003c c9d7 3c33 313e  ...2.....<..<31>
        0x0020:  4d61 7220 2031 2031 353a 3332 3a32 3320  Mar..1.15:32:23.
        0x0030:  6d69 6b72 6f74 696b 2020 2020 2052 6f75  mikrotik.....Rou
        0x0040:  7465 7220 3d20 3137 322e 3139 2e31 2e31  ter.=.172.19.1.1
13:32:23.048467 IP (tos 0x0, ttl 64, id 55671, offset 0, flags [DF], proto UDP (17), length 84)
    172.19.3.1.59825 > 172.19.3.50.5140: UDP, length 56
        0x0000:  4500 0054 d977 4000 4011 02c8 ac13 0301  E..T.w@.@.......
        0x0010:  ac13 0332 e9b1 1414 0040 18a6 3c33 313e  ...2.....@..<31>
        0x0020:  4d61 7220 2031 2031 353a 3332 3a32 3320  Mar..1.15:32:23.
        0x0030:  6d69 6b72 6f74 696b 2020 2020 2044 6f6d  mikrotik.....Dom
        0x0040:  6169 6e2d 5365 7276 6572 203d 2031 2e31  ain-Server.=.1.1
        0x0050:  2e31 2e31                                .1.1
13:33:21.967907 IP (tos 0x0, ttl 64, id 58436, offset 0, flags [DF], proto UDP (17), length 114)
    172.19.3.1.59825 > 172.19.3.50.5140: UDP, length 86
        0x0000:  4500 0072 e444 4000 4011 f7dc ac13 0301  E..r.D@.@.......
        0x0010:  ac13 0332 e9b1 1414 005e 9195 3c33 313e  ...2.....^..<31>
        0x0020:  4d61 7220 2031 2031 353a 3333 3a32 3120  Mar..1.15:33:21.
        0x0030:  6d69 6b72 6f74 696b 204f 7370 664e 6569  mikrotik.OspfNei
        0x0040:  6768 626f 7220 7b20 726f 7574 6572 2d69  ghbor.{.router-i
        0x0050:  643a 2031 3732 2e31 392e 3330 2e32 2073  d:.172.19.30.2.s
        0x0060:  7461 7465 3a20 4675 6c6c 207d 2068 656c  tate:.Full.}.hel
        0x0070:  6c6f                                     lo

ericwong3 commented 4 months ago

~Can confirm the issue is still present as of now, with the latest Graylog Open 6.0.4 and RouterOS 7.14.3~

Edit: After more searching, I found out the true culprit to be in BSD syslog's (RFC3164) design. At its heart, BSD syslog format's date field does not include time zone information. As a result, if your log source has a positive time offset and the Graylog instance is in UTC, the log will be registered for the future. (e.g. it is 8pm here in UTC+8, the log source sends 8pm without TZ in BSD syslog, Graylog treats it as 8pm UTC, but when fetching current logs 12nn UTC logs are shown, and thus the logs cannot be shown)

Solution There are two possible solutions if you have to use BSD syslog:

Configure your log source/server's timezone to be UTC. This imo is recommended as all systems' UTC = universal time and less ambiguity.
In the concerned Graylog's input settings, configure the Time Zone field to match your log source/server's timezone setting.

Otherwise you may also:

If your system supports IETF syslog (RFC5424), use it. It supersedes BSD syslog, and importantly in this regards it uses ISO-8601 timestamp format, hence no more timezone issue.
However, for Mikrotik, from my interpretation of the tcpdump, bsd-syslog=no is not IETF syslog but raw log (it does not even have the ISO-8061 timestamp at the beginning, let alone the other fields formats) hence in this case you might consider switching to "Raw/Plaintext UDP" input altogether.

Further reads: https://sematext.com/blog/what-is-syslog-daemons-message-formats-and-protocols/ https://graylog.org/post/time-zones-a-loggers-worst-nightmare/

Graylog2 / graylog2-server