grafana / grafana-operator

An operator for Grafana that installs and manages Grafana instances, Dashboards and Datasources through Kubernetes/OpenShift CRs
https://grafana.github.io/grafana-operator/
Apache License 2.0
863 stars 384 forks source link

[Bug] GrafanaAlertRuleGroup not deploying after Folder, Reconciler does not run for Alert Rule Groups #1593

Closed josemrs closed 2 months ago

josemrs commented 2 months ago

Describe the bug When a folder is created by GrafanaDashboard the GrafanaFolder does not actually manages it, the UID's in Grafana Cloud and K8S resource don't match.

When a GrafanaAlertRuleGroup and the GrafanaFolder are deployed the folder resource is created, but, the Alert fails and does not resync.

To Reproduce

  1. Create a helm chart deploying:
  1. Deploy the Helm chart (we use Spinnaker)

Expected behaviour The GrafanaFolder takes ownership of the folder.

josemrs commented 2 months ago

If you deploy a dashboard like this:

apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDashboard
metadata:
  name: testDashboard
  namespace: test
spec:
  allowCrossNamespaceImport: true
  folder: testFolder
  instanceSelector:
    matchExpressions:
    - key: grafanaStack
      operator: In
      values:
      - myStack
  json: |-
    {
    "annotations": {
      "list": [
        {
          "builtIn": 1,
          "datasource": {
            "type": "grafana",
            "uid": "-- Grafana --"
          },
          "enable": true,
          "hide": true,
          "iconColor": "rgba(0, 211, 255, 1)",
          "name": "Annotations & Alerts",
          "type": "dashboard"
        }
      ]
    },
    "editable": true,
    "fiscalYearStartMonth": 0,
    "graphTooltip": 0,
    "links": [],
    "panels": [],
    "schemaVersion": 39,
    "tags": [],
    "templating": {
      "list": []
    },
    "time": {
      "from": "now-6h",
      "to": "now"
    },
    "timepicker": {},
    "timezone": "browser",
    "title": "New dashboard",
    "version": 0,
    "weekStart": ""
    }

and then you deploy a folder like this:

apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaFolder
metadata:
  name: testFolder
  namespace: test
spec:
  allowCrossNamespaceImport: true
  instanceSelector:
    matchLabels:
      grafanaStack: myStack
  resyncPeriod: 5m
  title: Test Folder

You should end up in the same scenario we are.

Check the Grafana Cloud URL of the folder and the GrafanaFolder UID, you'll see they don't match.

In out case, for instance:

https://domain.grafana.net/dashboards/f/7b9d006d-22ad-4593-b72e-xxx/

VS

uid: f2ebc89d-c697-4ca3-8f8e-xxx

If now you try to deploy a GrafanaAlertRule with folderRef: testFolder it will report folder UID not found.

This issue links to this and asks for a fix either in the GrafanaFolder handling of already created folders or GrafanaDashboard to have folderRef instead, or in addition to, folder.

theSuess commented 2 months ago

This should be resolved by #1584. I'll close this as a duplicate of #1578. Feel free to reopen the other issue if this continues to be an issue after the next release