Prometheus with transformation

[DanielCastronovo]

Hello,

I want to use a transformation for Prometheus, for loop over every job labels to create check.

But i've tested with :

• template
• jsonPath
• javascript

And any of this works :)

Ex :

apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: apps
spec:
  interval: 30
  prometheus:
    - url: http://prometheus-operator-prometheus.kube-system.svc.cluster.local:9090
      name: apps
      query: up{namespace!~"kube-system|monitoring"} == 0
      transform:
        template: |
          {{ range .results }}
            Job: {{ .metric.job }} Value: {{ .value }}
          {{ end }}

If i use display it's works fine with the same template. Ex of result: Job: node-exporter Value: 15

But with transformation, no.

Error generated : transformation failure: unexpected end of JSON input

[moshloop]

The transformation response is expecting either a JSON object (e.g. to change labels/duration/metrics) or a JSON array of of objects - See https://canarychecker.io/concepts/expressions/transforms for the fields in the object to be returned

This will create a check per Job:

apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: prometheus-jobs
spec:
  interval: 30
  prometheus:
    - name: Jobs
      query: up{namespace!~"kube-system|monitoring"}
      url: http://prometheus.monitoring.svc:9090
      transform:
        expr: |
          dyn(results).map(r, {
            'name': r.job,
            'namespace': 'namespace' in r ? r.namespace : '',
            'labels': r.omit(["value", "__name__"]),
            'pass': r["value"] > 0
          }).toJSON()

And this will only create failing checks - more akin to an alert:

apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: prometheus-failing-jobs
spec:
  interval: 30
  prometheus:
    - name: Jobs
      query: up{namespace!~"kube-system|monitoring"} == 0
      url: http://prometheus.monitoring.svc:9090
      transform:
        expr: |
          dyn(results).map(r, {
            'name': r.job,
            'namespace': 'namespace' in r ? r.namespace : '',
            'labels': r.omit(["value", "__name__"]),
            'pass': false
          }).toJSON()

[DanielCastronovo]

Fine, thanks for your help. It's could be nice to add this example on the Prometheus check :)