banzaicloud / spark-metrics

Spark metrics related custom classes and sinks (e.g. Prometheus)
Apache License 2.0
175 stars 64 forks source link

Adding the ability to set custom labels on metrics #85

Open jnovick opened 1 year ago

jnovick commented 1 year ago

Is your feature request related to a problem? Please describe. When creating batches of jobs, I want the batch number to be a label on the same metric not a part of the metric name. This will make it easier to report both individual batches and all batches over a given time.

Describe the solution you'd like to see Something like this: https://github.com/banzaicloud/spark-metrics/issues/19#issuecomment-417202520

If you look at the Prometheus best practices, specifically the documentation about the data model and the part about Metric and label naming, you'll see that Prometheus advocates using more generic metric names

I know I can set labels statically using *.sink.<your-sink>.labels=label1=value1,label2=value2 but I want to apply labels where the value is determined at runtime.

Describe alternatives you've considered I can continue to make the metric name longer by adding to the accumulator, but it would be more in line with Prometheus best practices to use job labels.

Additional context None that I can think of