kubernetes-sigs / prometheus-adapter

An implementation of the custom.metrics.k8s.io API using Prometheus
Apache License 2.0
1.9k stars 551 forks source link

Is there a way to define default for a metric by default if not present #584

Open varunvijayagopal opened 1 year ago

varunvijayagopal commented 1 year ago

We use istio_requests_total for scaling . However this metric comes into picture only when we try to hit endpoint of service . /stats/proemetheus endpoint doesn't give this metric by default . Since we use this hpa , it comes as UNKNOWN till that service is accessed . I tried to add vector(0) so that if not present it gives a counter of 0 and this works fine on prometheus but still prometheus adapter complains with 404

Below is the query which we ar esuing

seriesQuery: istio_requests_total{destination_service_name!="",reporter="destination", destination_workload_namespace!=""} resources: overrides: destination_service_name: { resource: "service" } destination_workload_namespace: { resource: "namespace" } name: matches: "^(.*)_total" as: "${1}_per_second" metricsQuery: 'sum(rate(<<.Series>>{reporter="destination", <<.LabelMatchers>>}[1m]) or vector(0)) by (<<.GroupBy>>)'

dgrisonnet commented 1 year ago

/kind support /triage accepted /assign

huguesgr commented 9 months ago

I'm interested in this, have you found a workaround?

dgrisonnet commented 9 months ago

I don't think this was ever fixed

huguesgr commented 9 months ago

I found something that works in my use-case:

metricsQuery: <<.Series>>{<<.LabelMatchers>>} or clamp_max(absent(<<.Series>>{<<.LabelMatchers>>}),0)