repeatedly / fluent-plugin-multi-format-parser

Multi format parser for Fluentd
108 stars 16 forks source link

use multiline parser #13

Open TaLoN1x opened 5 years ago

TaLoN1x commented 5 years ago

I am trying to use multiline parser within this plugin and It seems that doesnt work

Ive tested it also with Java stack trace example from documentation...

My configuration:

<source>
  @type tcp
  tag "sometag.#{Socket.gethostname}"
  port 1234
  <parse>
    @type multi_format
    <pattern>
      format regexp
      expression /\<(?<pri>[0-9]+)\>(?<time>[^ ]* {1,2}[^ ]* [^ ]*) (?<host>[^ ]*) (?<message>.*)$/
      time_format %b %d %H:%M:%S
    </pattern>
    <pattern>
      format regexp
      expression /\<(?<pri>[0-9]{1,3})\>(?<time>[^ ]+) (?<host>[!-~]{1,255}) (?<message>.*)$/
      time_format %Y-%m-%dT%H:%M:%S.%L%z
    </pattern>
    <pattern>
      format regexp
      expression /\<(?<pri>[0-9]{1,3})\>(?<time>[^ ]+) (?<host>[!-~]{1,255}) (?<message>.*)$/
      time_format %Y-%m-%dT%H:%M:%S%z
    </pattern>
    <pattern>
      format multiline
      format_firstline /\d{4}-\d{1,2}-\d{1,2}/
      format1 /^(?<time>\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2}) \[(?<thread>.*)\] (?<level>[^\s]+)(?<message>.*)/
    </pattern>
  </parse>
</source>
repeatedly commented 5 years ago

See https://github.com/repeatedly/fluent-plugin-multi-format-parser#note If your logs are seperated into multiple records, this plugin doesn't work. You need to use concat plugin before.

akshayubale1 commented 3 years ago

If any one got the solution to use the multiline parse can you please share