fluent / fluent-plugin-prometheus

A fluent plugin that collects metrics and exposes for Prometheus.
Apache License 2.0
258 stars 79 forks source link

[question] how to grep key value and increment counter #102

Open kid101 opened 5 years ago

kid101 commented 5 years ago

{ "foo": "some value", "bar": "some value", "baz": "some value" }

In the readme it shows the key as foo and the plugin will count the message that contains foo key and increment the counter. How do I grep the value of foo to find hello, and if hello exists then increase the counter?

i5okie commented 4 years ago

need something similar.

my filter:

<filter mylabel>
  @type prometheus
  <metric>
    name router_codes
    type counter
    desc Counter of errors from Heroku Router
    <labels>
      tag ${tag}
      code ${router.code}
      dyno ${router.dest_dyno}
    </labels>
  </metric>
</filter>

Our logs are a mixed bag of data (heroku logs). Need to count on records that exist only. like only if the key exists. if it doesn't exist it just floods the fluentd logs, and pollutes prometheus data:

2020-07-29 20:03:20 +0000 [warn]: unknown placeholder `${router.code}` found
2020-07-29 20:03:20 +0000 [warn]: unknown placeholder `${router.dest_dyno}` found
2020-07-29 20:03:20 +0000 [warn]: unknown placeholder `${router.code}` found
2020-07-29 20:03:20 +0000 [warn]: unknown placeholder `${router.code}` found
2020-07-29 20:03:20 +0000 [warn]: unknown placeholder `${router.code}` found
2020-07-29 20:03:20 +0000 [warn]: unknown placeholder `${router.code}` found
2020-07-29 20:03:21 +0000 [warn]: unknown placeholder `${router.code}` found

image

bradbeattie commented 2 years ago

Is there a pre-existing solution to this in 2021, or should I look at writing a pull request?