Tr4sK
commented
8 years ago Hi, I’m experiencing the same issue: My input:
input { tcp { port => 1515 type => "nxlog" codec => json { charset => CP1252 } } }
Warn logstash.log: {:timestamp=>"2016-01-29T11:48:45.832000+0100", :message=>"Received an event that has a different character encoding than you configured.", :text=>"{\"EventReceivedTime\":\"2016-01-29 11:47:40\",\"SourceModuleName\":\"appl\",\"SourceModuleType\":\"im_file\",\"nxlog_message\":\MESSAGE",\"nxlog_tags\":\"appl\",\"nx log_type\":\"batch\",\"nxlog_hostname\":\"server.test\",\"nxlog_filename\":\"/opt/appl/FILE.log\"}", :expected_charset=>"UTF-8", :level=>:warn}
(This issue was originally filed by @german23 at https://github.com/elastic/logstash/issues/1857)
Hi,
i just encoutered a small bug regarding the json codec.
The default charset for this is "UTF-8" but regarding to the manual you can set it to other things like "CP1252" which is necessary for us cause we are using NXLog.
Configuring the other charset this way: tcp { port => 590 type => winevent codec => json { charset => "CP1252" } } unfortunately dont work. Regarding to the logs which i get:
{:timestamp=>"2014-10-07T17:32:57.293000+0200", :message=>"Received an event that has a different character encoding than you configured.", :text=>"{\"EventTime\":\"2014-10-07 17:32:56\",\"EventTimeWritten\":\"2014-10-07 17:32:56\",\"Hostname\":\"XXXXX\",\"EventType\":\"AUDIT_SUCCESS\",\"SeverityValue\":2,\"Severity\":\"INFO\",\"SourceName\":\"Security\",\"FileName\":\"Security\",\"EventID\":576,\"CategoryNumber\":2,\"Category\":\"An-/Abmeldung \",\"RecordNumber\":62703614,\"Domain\":\"XXXXX\",\"AccountName\":\"XXXX\",\"AccountType\":\"User\",\"Message\":\"timezone:+0\nBesondere Rechte bei neuer Anmeldung:\r\n \tBenutzername:\tXXXXX\r\n \tDom\xE4ne:\t\tXXXXXXX\r\n \tAnmeldekennung:\t\t(0x0,0xAC900BF)\r\n \tBerechtigungen:\tSeSecurityPrivilege\r\n\t\t\tSeBackupPrivilege\r\n\t\t\tSeRestorePrivilege\r\n\t\t\tSeTakeOwnershipPrivilege\r\n\t\t\tSeDebugPrivilege\r\n\t\t\tSeSystemEnvironmentPrivilege\r\n\t\t\tSeLoadDriverPrivilege\r\n\t\t\tSeImpersonatePrivilege \",\"EventReceivedTime\":\"2014-10-07 17:32:57\",\"SourceModuleName\":\"in\",\"SourceModuleType\":\"im_mseventlog\"}\r", :expected_charset=>"UTF-8", :level=>:warn}
the json input still expects the UTF-8 charset.
I also tried setting the charset in the codec module itself (logstash/lib/logstash/codec/json.rb)
config :charset, :validate => ::Encoding.name_list, :default => "CP1252"
and restarting the logstash instance but with no success.
It sounds like a bug.
Cheers