search

tagomoris / fluent-plugin-parser

Other
74 stars 23 forks source link

(Mistake) Fix forward error: invalid byte sequence in UTF-8 #1

Closed sonots closed 11 years ago

sonots commented 11 years ago

Hi, @tamogoris -san.

I met with an error when fluent-plugin-parser received kinds of malformed unicode character. 

2013-02-14 14:54:04 +0900: forward error: invalid byte sequence in UTF-8
  2013-02-14 14:54:04 +0900: /usr/lib/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-plugin-parser-0.2.1/lib/fluent/plugin/fixed_parser.rb:21:in `match'
  2013-02-14 14:54:04 +0900: /usr/lib/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-plugin-parser-0.2.1/lib/fluent/plugin/fixed_parser.rb:21:in `call'
  2013-02-14 14:54:04 +0900: /usr/lib/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-plugin-parser-0.2.1/lib/fluent/plugin/fixed_parser.rb:235:in `parse'
  2013-02-14 14:54:04 +0900: /usr/lib/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-plugin-parser-0.2.1/lib/fluent/plugin/out_parser.rb:75:in `block in emit'
  2013-02-14 14:54:04 +0900: /usr/lib/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.31/lib/fluent/event.rb:127:in `feed_each'
  2013-02-14 14:54:04 +0900: /usr/lib/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.31/lib/fluent/event.rb:127:in `each'

One of our applications outputted an error log like malformed or illegal unicode character in string [? 遥《K], cannot convert to JSON, so, right, we can not convert the character to JSON.

This patch is the simplest way to fix the error. What do you think?