search

mozilla-services / hindsight

Hindsight - light weight data processing skeleton
Mozilla Public License 2.0
665 stars 55 forks source link

Issue while decoding syslog #195

Open KalaiarasiAyyadurai opened 5 years ago

KalaiarasiAyyadurai commented 5 years ago

I had installed hindsight to parse the syslogs using lua modules(lpeg.sub_decoder_util and decoders.syslog).It reads the log and writes the decoded log in the output.Decoded log contains some junk characters as shown below.

Configuration: filename = "file.lua" input_filename = "syslog.log" send_decode_failures = true decoder_module = "decoders.syslog"

decoders_syslog = { template = "%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%", }

Input(syslog.log): Feb 13 14:25:19 ubuntu sshd[7192]: Accepted publickey for foobar from 216.160.83.56 port 4242 ssh2

Output: ▒d▒e5J▒▒\H▒▒Mi▒▒▒▒▒Ѽ▒" input.file2?Accepted publickey for foobar from 216.160.83.56 port 4242 ssh2@▒8JubuntuR

Please, help to fix.

trink commented 5 years ago

Where is that output being captured from? If it is from an output plugin, it looks like it is using a Heka protobuf encoder you can switch to heka_debug for human readable output. You can also examine the internal queue using lsb_heka_cat <output_path>/input/0.log

KalaiarasiAyyadurai commented 5 years ago

Thanks for your direction Trink.The output is being captured from input plugin.I got your message on switching the encoder.I will try this and revert back of I have any issues.