fluent / fluent-plugin-prometheus

A fluent plugin that collects metrics and exposes for Prometheus.
Apache License 2.0
258 stars 79 forks source link

Parsing "type" in metric block fails if using Ruby syntax #149

Open jonmoter opened 4 years ago

jonmoter commented 4 years ago

I'm using the ruby-style syntax to define fluentd configuration. I have a config block that looks like:

  match('output.**') do
    type :copy

    store {
      type :prometheus

      metric {
        name :fluentd_output_status_num_records_total
        type :counter
        desc 'The total number of outgoing records'
        labels {
          tag '${tag}'
        }
      }
    }
  end

But when I try running the code, I get the error:

config error file=\"config/fluent.conf.rb\" error_class=Fluent::ConfigError error=\"type option must be 'counter', 'gauge', 'summary' or 'histogram'\"

I hunted down the line of code that was generating that error:

https://github.com/fluent/fluent-plugin-prometheus/blob/b1421b5883d44c78ade1dadbeb76aaefe0cfab27/lib/fluent/plugin/prometheus.rb#L78

I tweaked the error message that it output to write the keys of the element hash. I got:

type option must be 'counter', 'gauge', 'summary' or 'histogram', element keys: name,@type,desc

As far as I can tell, the Fluent::Config::Element class has some magic in it that turns type into @type as a hash key.

ganmacs commented 4 years ago

Right. type is one of reserved keywords in fluentd. BTW, we can't maintain Ruby DSL configuration and will make it deprecated because we don't have enough resources to maintain it. We recommend that you use the original format of fluentd.

https://github.com/fluent/fluentd/blob/a5badcdf916bebcce25af5aac21ad13ce1c3f9a1/lib/fluent/config/dsl.rb#L100

<match output.**>
  @type copy
  <store>
    @type prometheus
    <metric>
      name fluentd_output_status_num_records_total
      type counter
      desc "The total number of outgoing records"
        <labels>
          tag "${tag}"
      </labels>
    </metric>
  </store>
</match>