Prometheus and Grafana Integration

[ionfury]

It would be great if the operator included or provisioned resources to integrate with Prometheus and Grafana out of the box. Specifically, creating the following resources would be very helpful:

• Either podMonitor or serviceMonitors for downstream dragonfly metrics & the operator itself.
• PrometheusRules to alert on issues with the operator & downstream dragonflies.
• A configMap to automatically create grafana dashboard(s) for the above monitoring.

[Pothulapati]

This is a valid feature request @ionfury! Don't have a timeline for it yet but might work on it next week! Meanwhile, feel free to raise a PR if you want/can!

[hermanbanken]

Plus one here too. We use DataDog with OpenMetrics so this means that the Pod instance that runs DragonflyDB should have certain DataDog-specific annotations (v1):

    annotations:
      ad.datadoghq.com/mycontainer-dragonfly.check_names: '["openmetrics"]'
      ad.datadoghq.com/mycontainer-dragonfly.init_configs: '[{}]'
      ad.datadoghq.com/mycontainer-dragonfly.instances: '[{
          "prometheus_url": "http://%%host%%:6379/metrics",
          "namespace": "dragonflydb",
          "metrics": ["*"]
          }]'

[Pothulapati]

@hermanbanken Ah, We can have a way to pass annotations into the Dragonfly pod! This shouldn't be hard. Just adding an annotations field into the CRD and passing that into the Statefulset spec.

[hermanbanken]

@Pothulapati that would be great!

[Pothulapati]

Fixed with https://github.com/dragonflydb/dragonfly-operator/pull/101 https://github.com/dragonflydb/dragonfly-operator/pull/113