search

tagomoris / fluent-plugin-parser

Other
74 stars 23 forks source link

How to track the rejected log lines? #31

Closed vishnuvgdemos closed 8 years ago

vishnuvgdemos commented 8 years ago

We are using fluentd - parser plugin to parse a log file. Basically it uses regex pattern to extract the values from log file.. But because of the invalid input log data, frequent data loss happening. Is there anyway to track the mismatched data. Right now the parser is parsing only the matching regex values.

We are not aware of how to track the mismatched lines. Just like a way to log the mismatched data or show a warning. The fluentd tail plugin not showing any warning even when it encounters the mismatched data Someone having any idea. Please suggest.

vishnuvgdemos commented 8 years ago

Hi, I got the non matching logs, when I turned on "suppress_error_message" to false.(by default). It wil be helpful to add functionality to route the non matching logs as pointed out in #23. Thanks

tagomoris commented 8 years ago

I think your problem is already solved (#23 is another problem). Please reopen if not.