search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.22k stars 1.05k forks source link

Wrong message parsing with rsyslog #16

Closed dnauck closed 13 years ago

dnauck commented 13 years ago

Hello,

i've an existing rsyslog infrasturcture. Many clients send their messages to a central rsyslog server with one of the "rsyslog methods". On this rsyslog server is graylog2 installed and listening on localhost for incoming messages. The rsyslog server forwards all messages in the _RSYSLOGTraditionalForwardFormat [1] to the local graylog2 server.

It looks like graylog2 is parsing the input in a wrong way (or the rsyslog message format template is wrong?).

Example:

rsyslog sends following message (received from the "jabber" host) to the local graylog2:

 <38>Nov 29 23:19:46 jabber sshd[27089]: last message repeated 2 times

But the message is shown in the graylog2 interface as:

Date: 29.11.2010 - 23:19:46
Host: localhost
Severity: Informational
Facility: Unknown
Message: Nov 29 23:19:46 jabber sshd[27089]: last message repeated 2 times

[1] http://www.rsyslog.com/doc/rsyslog_conf_templates.html

lennartkoopmann commented 13 years ago

Both your issue reports will unfortunately not make it into the next release.

There will be a workaround for the facility problem: The next release (maybe released today) will allow you to overwrite and add own facilities.

lennartkoopmann commented 13 years ago

FYI: I've moved both your issue reports to the top of the list.

lennartkoopmann commented 13 years ago

I just committed a patch for this. I'll upload a BETA release of the server with the patch included in a few minutes. Could you try it out?

If it makes no problems, it will become an official release.

(This should also fix the wrong facility - Needs also some testing by users)

lennartkoopmann commented 13 years ago

The BETA release caused some trouble, so I deleted it again. There now is a stable patchlevel release that fixes the sometimes wrong facility numbers for syslog. This bug gets my attention in the next days again as it is more complicated. Thanks!

philgh commented 13 years ago

I have a similiar setup here (one big rsyslog receiving all kinds of logs). Now rsyslogd forwards them locally to graylog2-server.

So far I haven't had a problem with facilities but the more pressing one is definitely that "host" is always "localhost" and the hostname part of the message seems to be ignored.

I had a quick look at the syslog4j library used. Could it be solved by checking "isHostStrippedFromMessage" and use the portion of message if it returns false? http://www.syslog4j.org/docs/javadoc/org/productivity/java/syslog4j/server/SyslogServerEventIF.html#isHostStrippedFromMessage%28%29

Nevertheless I must say this is such a nice software already! Can't wait to see it evolve ;)

philgh commented 13 years ago

Small correction: This method was introduced in syslog4j version 0.9.42 (10/29/2010) - so it's not available in the bundled version.

I tried to build graylog2-server with a more recent syslog4j but it breaks the syslog thread somehow.

I was hoping to see my "host name issue" get fixed by this change in syslog4j:

Version 0.9.41 (10/28/2010)
+ Improved SyslogServerEvent for better host address and host name
  matching/parsing
lennartkoopmann commented 13 years ago

FYI: http://lists.serverlocation.com/pipermail/syslog4j-support/2011-January/000104.html

Working on this.

bracki commented 13 years ago

A possible workaround can be found here: https://github.com/Jimdo/graylog2-server/tree/graylog2-server-0.9.3-jimdo

You need to enable StructuredSysLogServerEvents. There the host parsing works alright.

lennartkoopmann commented 13 years ago

Keep in mind that this is version 0.9.3 and not compatible with 0.9.4 web interface

lennartkoopmann commented 13 years ago

Quick update from me: I'm currently in contact with the syslog4j author and should be able to release a patched package tomorrow!

dnauck commented 13 years ago

https://github.com/Graylog2/graylog2-server/pull/37

mikewaters commented 13 years ago

For future reference, a functional >= graylog2-server-0.9.5p1 forwarded-from-rsyslog output template looks like:

/etc/rsyslog.conf:

$template Graylog2Friendly,"<%PRI%> %TIMESTAMP% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n"
$ActionForwardDefaultTemplate Graylog2Friendly

/etc/rsyslog.d/example:

*.* @@graylog2.domain.tld

None of the stock templates seemed to get it right.

lennartkoopmann commented 13 years ago

@rmwsoft - Thank you! I'll add that to the wiki. There are some people on the mailing list possibly waiting for this solution.