Additional metadata

[ldaneliukas]

We're looking for utilizing exporter_exporter to expose additional metadata about each module that would then be consumed by third-party systems via the JSON response.

Currently, what we're using is labels. I.e. each module has a section of labels that is also exposed via JSON and can then be used by another system which constructs scrape targets for file_sd. Note, these aren't actually added to the metrics themselves by exporter_exporter as it still only acts as a proxy. Would that be something that could be added to exporter_exporter?

E.g.

modules:
  node:
    method: http
    http:
       port: 9100
    labels:
       labelname: labelval
       labelname2: labelval2

and

type moduleConfig struct {
    Method  string                 `yaml:"method"`
    Timeout time.Duration          `yaml:"timeout"`
    Labels  map[string]string      `yaml:"labels"`

That's pretty much it, we're not actually doing anything more than exposing it in the json response.

[tcolgate]

Since they aren't really used by the exporter itself, it does feel a little odd, but I can see some potential uses. My gut instinct is to rename it "metadata", and allow any arbitrary map[string]interface{} to be set. Unless there is some other way of making use of them I've not considered (maybe exposing them through the exporters own metrics?). It does feel like this should probably live elsewhere though. e.g. if you are mostly trying to get info about the node the exporter is running on, you probably would be better using a file being read and exposed via the node exporter).

[ldaneliukas]

Yeah, that's exactly why I'm asking first. I also thought that naming it metadata would be far better suited for general use and wouldn't mislead people.

As for the use case, we'd be adding the metadata to individual modules, not the whole config for expexp.

For us, this allows us to inject those labels to all metrics of any exporter running on the machine. Helps with tagging services, components, tenants, etc in a multi-tenant use case where Prometheus is shared by many teams that use similar exporters.

I can whip up a PR for metadata as discussed if that's something you see being added.

[tcolgate]

Here's my thinking (I think). If you are going to use this to join some labels on to the timeseries later, then probably what you want is a metric that gathers those labels (at least, this is one way of doing it, similar to what kube-state-metrics does). So you'd have: expexp_module_labels_unit{module="mymodule" label1="1" label2="2"} = 1 You're either making something like that yourself elsewhere (maybe not literally a metrics, but something like that), and either doing the join elsewhere, or in prom). Now, if you can write the labels/metadata to the expexp config, you can write it anywhere, and if you can write it anywhere you could just write that metrics directly into a file for node_exporter to push out, then you don't need the feature in expexp at all.

I can still be persuaded, and obviously I don't know your use case.

[tcolgate]

Going to close this, I'm not sure this would add enough value.

[sebinjohn]

I came searching for this exact requirement (ie. having the ability to add labels).

Please let me know if I should create a new issue?

Here is my use case

I have two environments for an application

• staging
• production

Two scrape jobs are then created on prometheus side to scrape both environment.

by supporting the addition of labels, on the expexp side i will be able to specify the environment from which the metrics are exported. This will remove the need for configuring per environment per application prometheus jobs.