search

rsyslog / rsyslog

a Rocket-fast SYStem for LOG processing
http://www.rsyslog.com
GNU Lesser General Public License v3.0
2.01k stars 647 forks source link

handle BOM more inteligently #284

Open rgerhards opened 9 years ago

rgerhards commented 9 years ago

Inf RFC5424 messages, BOM indicates that the message is indeed in Unicode. So this is more or less a flag. However, rsyslog does currently treat the BOM as a regular part of the message. That can cause problems with outputs.

We should handle BOM in a way that makes

a) the information that it is Unicode available as metadata b) remove the BOM from the actual message

Note that when forwarding via RFC5424 format, we need to check the "BOM-Flag" and include it in the forwarded message if needed. To work properly with the template engine, this means we need to create a new property which emits a BOM if the message is Unicode and otherwise an empty string.

ashera24 commented 9 years ago

Has any new version of rsyslog come to overcome this issue?Even I am prepending the message with BOM as per RFC ,but Rsyslog just prints as it is without extracting the metadata i.e message

ashera24 commented 9 years ago

Like below :

<13>1 2015-04-13T12:16:32.115Z localhost-1 TRAPMGR SNMPCfgTask - : traputil.c(777) 565 %% Cold Start: Unit: 0
nicklowe commented 9 years ago

This really ought to be changed as rsyslog isn't being friendly with its current behaviour.

nicklowe commented 9 years ago

I'm currently engaging with Aerohive to see if they can make their use of Syslog conform to RFC 5424 which would mean including a BOM. Issues like this don't help make the case for doing it sadly. It would be awesome if this could be fixed therefore. Cheers :)

luigiberrettini commented 8 years ago

I think this is a bug that makes RSyslog not RFC compliant. Are UTF8 messages without BOM handled correctly (is UTF8 detected some way)? If we add BOM (keeping this bug) detection is better handled?