[Question] Custom metrics tag

[mario45211]

Hi Jet Team!

I am currently working on exposing my ow metrics from a Jet pipeline using Prometheus as described in doc. Question - it is possible to include my custom tag to those metrics when exposing them via Prometheus endpoint to pass some additional context and make me query/group metrics by such tag ?

Example:

com_hazelcast_jet_Metrics_my_metric_name{instance="nervous_beaver",tag0="\"job=06ab-c7fc-af00-0001\"",tag1="\"exec=06ab-c7ff-5580-0001\"",tag2="\"vertex=PreValidate event\"",tag3="\"procType=TransformUsingServiceP\"",tag4="\"proc=2\"",tag5="\"user=true\"",tag6="\"my_tag=my_value\"",}

Example of expected API:

Metrics.metric("my_metric", Unit.COUNT, "my_tag=my_value").increment()

Best regards!

[mario45211]

For those who struggling with similar issue this is my workaround - I've concatenated my custom tag value into metric name and used metric_relabel_configs Prometheus configuration like:

# metric name: com_hazelcast_jet_Metrics_my_tag=my_value
metric_relabel_configs:
  - source_labels: [__name__]
    regex: 'com_hazelcast_jet_Metrics_my_tag=(.*)'
    replacement: '${1}'
    target_label: my_tag

  - source_labels: [__name__]
    regex: '(com_hazelcast_jet_Metrics)_my_tag=.*'
    replacement: '${1}'
    target_label: __name__
# final query: com_hazelcast_jet_Metrics{my_tag=my_value}

Above, first one extract my_value to my_tag label and second trim metric name, removing the my_tag=my_value part.