I was trying to parse logs from Amazon ELB and sometimes when I got this wierd string :
+---------------------------------------------------------+
logstash crashed with this error :
{:timestamp=>"2014-11-27T10:28:31.255000+0200", :message=>"+---------------------------------------------------------+\n| An unexpected error occurred. This is probably a bug. |\n| You can find help with this problem in a few places: |\n|
I then went to the IRC and spoke with a nice guy that also he had some problems with the UTF.
so I decided to do a little reaserch and I have noticed that the problem is actually is in the illegal charachters.
this was his line :
<133>Nov 26 08:31:34 XXX squid-access-log: 1416983487.192 0 10.X.X.X NONE/400 3533 \u0016\u0003 b*\xAA~ - NONE/- text/html
and then I've found this :
filter {
mutate {
gsub => [
# replace all forward slashes with underscore
"fieldname", "/", "_",
```
# replace backslashes, question marks, hashes, and minuses with
# dot
"fieldname2", "[\\?#-]", "."
]
```
}
}
this is an example taken from :
http://logstash.net/docs/1.4.2/filters/mutate
then I tried to parse the two patterns without no problems ! and yes...it did change our log a little big but if to be honest we preffer the current sitaution then the failures we had before.
I believe that it will solve your errors as well, however I suggest to check if the output looks good for you at the end of the day.
Enjoy guys !
suyograo
commented
9 years ago
Hi everyone,
I was trying to parse logs from Amazon ELB and sometimes when I got this wierd string : +---------------------------------------------------------+
logstash crashed with this error : {:timestamp=>"2014-11-27T10:28:31.255000+0200", :message=>"+---------------------------------------------------------+\n| An unexpected error occurred. This is probably a bug. |\n| You can find help with this problem in a few places: |\n|
I then went to the IRC and spoke with a nice guy that also he had some problems with the UTF. so I decided to do a little reaserch and I have noticed that the problem is actually is in the illegal charachters.
this was his line :
<133>Nov 26 08:31:34 XXX squid-access-log: 1416983487.192 0 10.X.X.X NONE/400 3533 \u0016\u0003 b*\xAA~ - NONE/- text/html and then I've found this : filter { mutate { gsub => [ # replace all forward slashes with underscore "fieldname", "/", "_", ``` # replace backslashes, question marks, hashes, and minuses with # dot "fieldname2", "[\\?#-]", "." ] ``` } } this is an example taken from : http://logstash.net/docs/1.4.2/filters/mutate then I tried to parse the two patterns without no problems ! and yes...it did change our log a little big but if to be honest we preffer the current sitaution then the failures we had before. I believe that it will solve your errors as well, however I suggest to check if the output looks good for you at the end of the day. Enjoy guys !