Add ignore_key_not_exist option to filter_parser.

osamunmun commented 8 years ago

If 'key_name' doesn't exist, I got "ArgumentError parser io argument must be a String or respond to readpartial() or read()." from Oj.load(nil) at line45 @parser.parse .

Adding "oj" to gemspec may not be good idea, I couldn't catch up with any good idea about testing the case of "oj".

tagomoris commented 8 years ago

Good catch! I have some points for this change:

IMO, this plugin should do these for missing specified key-values:

warn it in logs at first (ignore_key_not_exist suppress this warning)
just skip that record if reserve_data is NOT specified
emit the same record with input if reserve_data is specified

@osamunmun How do you think about updating your patch like this?

Adding "oj" to gemspec as development dependency is ok for me.
This plugin also has output plugin version, and these 2 are expected to work in same way. Could you add same change to out_parser.rb and its test?

osamunmun commented 8 years ago

@tagomoris I agree with your suggestion about warn, skip and emit same record. I have changed them, and added same code into out_parser.

tagomoris commented 8 years ago

Thanks! LGTM.

tagomoris commented 8 years ago

Released v0.6.1 with this change.

tagomoris / fluent-plugin-parser

Add ignore_key_not_exist option to filter_parser. #33