Adding init_label_set method from prometheus ruby client

[AntoineC44]

Hello,

Preview to get feedback, will add tests if accepted on principle.

Following https://github.com/fluent/fluent-plugin-prometheus/pull/180 here is a PR to add the init_label_set feature from prometheus ruby client (see https://github.com/prometheus/client_ruby/commit/0e0f17fb38f2e703f8adfaf16d75b26af32b5053)

Here is what it gives with this implem

fluent.conf

 <source>
    @type tcp
    format json
    bind "0.0.0.0"
    port 5170
    tag "test"
    <parse>
      @type json
    </parse>
  </source>
  <match test>
    @type prometheus
    <metric>
      name histogram
      type histogram
      desc histogram with current implem, not initialized until first event
      key test
      <labels>
        label a
      </labels>
    </metric>
    <metric>
      name histogram_initialized
      type histogram
      desc histogram initialized
      key test
      buckets 0, 1, 10, 1000
      <labels>
        iam initialized!
      </labels>
      <initlabels>
        iam initialized!
      </initlabels>
    </metric>
    <metric>
      name alert_counter_initalized
      type counter
      desc User knowing part of/all future values of reccordAccessor $.filename can initialize the corresponding counters
      <labels>
        filename $.filename
      </labels>
      <initlabels>
        filename foo.txt
      </initlabels>
      <initlabels>
        filename foo.tar.gz
      </initlabels>
    </metric>
    <metric>
      name simple_summary_initialized
      type summary
      desc Providing no initlabels in the block yields a simple initialized metric
      key test
      <initlabels>
      </initlabels>
    </metric>
  </match>
  <match>
    @type prometheus
    <metric>
      name counter
      type counter
      desc string labels must be identical between labels and initlabels blocks
      key test
      <labels>
        source fluentd
        filename $.filename
      </labels>
      <initlabels>
        source fluentd
        filename foo.tar.gz
      </initlabels>
      <initlabels>
        source fluentd
        filename foo.txt
      </initlabels>
    </metric>
  </match>
  <match>
    @type prometheus
    <labels>
      globallabel initvalue2
    </labels>
    <metric>
      name counter_with_global_label
      type counter
      desc Identically, global labels must be initialized in initlabels block
      <labels>
        locallabel $.key
      </labels>
      <initlabels>
        locallabel initvalue
        globallabel initvalue2
      </initlabels>
    </metric>
  </match>
  <source>
    @type prometheus
  </source>

result

$ curl http://localhost:24231/metrics
# TYPE histogram histogram
# HELP histogram histogram with current implem, not initialized until first event
# TYPE histogram_initialized histogram
# HELP histogram_initialized histogram initialized
histogram_initialized_bucket{iam="initialized!",le="0"} 0.0
histogram_initialized_bucket{iam="initialized!",le="1"} 0.0
histogram_initialized_bucket{iam="initialized!",le="10"} 0.0
histogram_initialized_bucket{iam="initialized!",le="1000.0"} 0.0
histogram_initialized_bucket{iam="initialized!",le="+Inf"} 0.0
histogram_initialized_sum{iam="initialized!"} 0.0
histogram_initialized_count{iam="initialized!"} 0.0
# TYPE alert_counter_initalized counter
# HELP alert_counter_initalized User knowing part of/all future values of reccordAccessor $.filename can initialize the corresponding counters
alert_counter_initalized{filename="foo.txt"} 0.0
alert_counter_initalized{filename="foo.tar.gz"} 0.0
# TYPE simple_summary_initialized summary
# HELP simple_summary_initialized Providing no initlabels in the block yields a simple initialized metric
simple_summary_initialized_sum 0.0
simple_summary_initialized_count 0.0
# TYPE counter counter
# HELP counter string labels must be identical between labels and initlabels blocks
counter{source="fluentd",filename="foo.tar.gz"} 0.0
counter{source="fluentd",filename="foo.txt"} 0.0
# TYPE counter_with_global_label counter
# HELP counter_with_global_label Identically, global labels must be initialized in initlabels block
counter_with_global_label{locallabel="initvalue",globallabel="initvalue2"} 0.0

Example of things that triggers configError with this implem

Having initlabels not matching a string value from label (metric would never be used)

  <match test>
    @type prometheus
    <metric>
      name counter
      type counter
      desc histogram with current implem, not initialized until first event
      <labels>
        label a
      </labels>
      <initlabels>
        label b ---> would trigger error because value a is expected on whole lifetime of metric
      </initlabels>
    </metric>
  </match>

Having initlabels not matching the signature of labels

  <match test>
    @type prometheus
    <metric>
      name counter
      type counter
      desc histogram with current implem, not initialized until first event
      <labels>
        somelabel a
        anotherlabel b
      </labels>
      <initlabels>
        somelabel a
        someinitlabel b ---> would trigger error because someinitlabel is not present in <labels> block, this has no meaning
      </initlabels>
    </metric>
  </match>

[AntoineC44]

Hello @repeatedly, would appreciate your feedback. Adds a layer of complexity but fully retrocompatible and can be useful. Cheers

[AntoineC44]

Hello @cosmo0920 @repeatedly, Kindly requesting some feedback from you guys on this feature and implementation. If ok for you I would add some dedicated tests, Have a good day, Antoine

[amaury-d]

Thanks @AntoineC44 for this pull request. I confirm this feature would be very useful too on our side.

[AntoineC44]

Hello @cosmo0920 @repeatedly @ashie, Sorry to ping you again, are you able to provide some feedback? If ok for you I would add some dedicated tests, Have a good day, Antoine

[polohuang]

hello, this is very useful for me is there any progress?

[AntoineC44]

Hello, I wanted some feedback with the implem from a maintainer before writing tests and having the PR ready, No feedback for now, I can retag @cosmo0920 @repeatedly @ashie, Thanks, Antoine

[ashie]

In addition, could you add description about new config parameter to README.md?

[AntoineC44]

Did the changes, it indeed ends-up being less configuration but more code, especially to handle corner cases around placeholders and reccord accessors.

[AntoineC44]

Fixed typo in README and not broad enough test for the config

[AntoineC44]

@ashie Hello any news?

[AntoineC44]

Hello @ashie, when is the next release planned? Thanks!