logstash-plugins / logstash-output-influxdb

Apache License 2.0
58 stars 77 forks source link

Inject event's fields into data_points #2

Closed zavyrylin closed 3 years ago

zavyrylin commented 9 years ago

It would be great to add a capability to get some fields matched a given pattern from event object and put them into data points. It is useful in the cases when your logs may contain some metrics, but their structure is dynamic/not known so it cannot be defined strictly inside the config of the logstash instance. The pull request adds some "dynamicity" to output configuration. This can be done by the new configuration options given in the next example:

output {
  influxdb {
    # all fields of event matched these regexps will be appended to data_points
    include_fields => ["host", "request\.(status|time)", "handler\.queue\d+"]

    # all fields of event will never become a part of data series
    exclude_fields => ["handler\.queue(3|5)"]

    # if some of event's fields you're interested in are hashes
    # this options will extract the deepest key-value pairs
    # "flattening" these hashes
    flatten_included_hashes => true
  }
}
brupm commented 9 years ago

@jordansissel is this something you consider a good idea? Should @zavyrylin write specs for his code?

elasticsearch-release commented 9 years ago

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run; then say 'jenkins, test it'.