grafana / helm-charts

Apache License 2.0
1.65k stars 2.27k forks source link

[grafana] Grafana failed to load dashboard from json error=EOF #764

Open mpavlovicbb opened 3 years ago

mpavlovicbb commented 3 years ago

Hi,

I am trying to import dasboards with helm installation but always getting following error

lvl=eror msg="failed to load dashboard from " logger=provisioning.dashboard type=file name=test file=/var/lib/grafana/dashboards/test/minio.json error=EOF

values file have following configuration:

dashboardProviders: dashboardproviders.yaml: apiVersion: 1 providers:

  • name: 'test' type: file disableDeletion: false editable: true allowUiUpdates: true options: path: /var/lib/grafana/dashboards/test

dashboards: test: minio: file: dashboards/minio.json

And file structure is following:

. ├── dashboards │   └── minio.json └── values.yaml

Afther initialising pod there is empty file minio.json on /var/lib/grafana/dashboards/test

Inistalation is done with command

helm upgrade --install -n grafana grafana grafana/grafana --values values.yaml

Is there any solution to this or I am missing simething?

qdupuy commented 2 years ago

Hello,

you have to declare your dashboard volume in the grafana container so that it can find your json files

chukynax commented 2 years ago

Hello,

you have to declare your dashboard volume in the grafana container so that it can find your json files

could you please clarify ?

roshan8 commented 2 years ago

I'm running into the same issue. Any workaround?

chukynax commented 2 years ago

i put json next to values and set variable during setup

dashboards:
  default:
    kubernetes:
      file: dashboards/kubernetes.json

another option is to put raw json in values

dashboards:
  default:
    kubernetes:
      json:|
        {}

i wanted to use helm charts repo just to set needed values in argocd, but for these workarounds i had to fetch untar chart itself and store it in own repo. all other tricks i tried with no luck

roshan8 commented 2 years ago

@chukynax Second option would make the file not so maintainable. Didn't get your first option though. Can you please explain what do you mean by JSON next to values?

chukynax commented 2 years ago

@roshan8 fetch untar grafana chart, then downloaded dashboard in json format and put it next to values.yaml in chart, in my example in directory: "dashboards"

roshan8 commented 2 years ago

oh got it. Thanks!

ewerton-silva00 commented 2 years ago

Hey, guys.

I have the same issue.

How are organized files:

.
├── dashboards
│   └── container-metrics.json
└── values.yaml

1 directory, 2 files

My values.yaml file:

rbac:
  create: true

adminUser: admin
adminPassword: strongpassword

persistence:
  type: pvc
  enabled: true
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  size: 10Gi

plugins:
  - grafana-clock-panel

datasources:
  datasources.yaml:
  apiVersion: 1
  datasources:
  - name: Prometheus
    type: prometheus
    url: http://prometheus.prometheus.svc.cluster.local:9090
    access: default
    isDefault: true

dashboardProviders:
  dashboardproviders.yaml:
    apiVersion: 1
    providers:
      - name: 'dojot'
        orgId: 1
        folder: 'Dojot'
        folderUid: ''
        type: file
        disableDeletion: false
        editable: false
        updateIntervalSeconds: 10
        allowUiUpdates: false
        options:
          path: /var/lib/grafana/dashboards/dojot

dashboards:
  dojot:
    container-metrics:
      file: dashboards/container-metrics.json

sidecar:
  datasources:
    enabled: true
  dashboards:
    enabled: true

I tried too without the sidecars, but I have the same result.

Command utilized to install the Helm Charts:

helm upgrade --install grafana grafana/grafana --version 6.20.3 --create-namespace --namespace grafana --values values.yaml

The result:

t=2021-12-20T12:31:28+0000 lvl=eror msg="failed to load dashboard from " logger=provisioning.dashboard type=file name=dojot file=/var/lib/grafana/dashboards/dojot/container-metrics.json error=EOF
t=2021-12-20T12:31:38+0000 lvl=eror msg="failed to load dashboard from " logger=provisioning.dashboard type=file name=dojot file=/var/lib/grafana/dashboards/dojot/container-metrics.json error=EOF
DucretJe commented 2 years ago

I have the same issue deploying with argocd using this repo (639fa1c) as source. My values.yaml:

        dashboardProviders:
          dashboardproviders.yaml:
            apiVersion: 1
            providers:
            - name: 'default'
              orgId: 1
              folder: ''
              type: file
              disableDeletion: false
              editable: true
              options:
                path: /var/lib/grafana/dashboards/default
        dashboards:
          default:
            kubernetes-cluster:
              gnetId: 315
              revision: 3
              datasource: Prometheus

I noticed while connecting to grafana pod and having a look in /var/lib/grafana/dashboards/default that my json file was actually empty. I looked in the logs of the download-dashboards init container and see nothing fishy

4+ set -euf
3+ mkdir -p /var/lib/grafana/dashboards/default
2+ curl -skf --connect-timeout 60 --max-time 60 -H 'Accept: application/json' -H 'Content-Type: application/json;charset=UTF-8' https://grafana.com/api/dashboards/315/revisions/3/download
1+ sed '/-- .* --/! s/"datasource":.*,/"datasource": "Prometheus",/g'
Swaps76 commented 2 years ago

Hello,

you have to declare your dashboard volume in the grafana container so that it can find your json files

Can you please clarify - I am getting the same issue. Thanks!

I am just digging around and wondering if this is where you mean to add the volume to the value.yaml?

## Additional grafana server volume mounts
# Defines additional volume mounts.
extraVolumeMounts: []
  # - name: extra-volume-0
  #   mountPath: /mnt/volume0
  #   readOnly: true
  #   existingClaim: volume-claim
  # - name: extra-volume-1
  #   mountPath: /mnt/volume1
  #   readOnly: true
  #   hostPath: /usr/shared/
ktomaszx commented 2 years ago

I see many question regarding this case so I will leave this comment to clarify why this importing dashboard config from json file doesn't work. Below code dashboards: {} default: custom-dashboard: file: dashboards/custom-dashboard.json does not mean that we are importing custom-dashboard.json file from workspace we are working in. It uses custom-dashboard.json file from chart repository. https://github.com/grafana/helm-charts/blob/main/charts/grafana/dashboards/custom-dashboard.json This is why configmap data: is empty. Really bad that is not described in values.yaml comments.

g-linville commented 2 years ago

I think @ktomaszx is right. I ran into this same problem. I recommend using --set-file with Helm to just insert raw JSON like this:

helm install releasename grafana/grafana -f my_values.yaml --set-file dashboards.default.dashboard-name.json=my-dashboard.json

Abuu001 commented 2 years ago

Open

For me actually I had the same problem for some weeks .

it was not easy to notice the cause but I had missed only one symbol in my dashboard.json file . After weeks of debugging I actually found out my dashboard.json file had a missing } (closing brace) at the end of file.

trydalch commented 2 years ago

@ktomaszx So is there any way to load a file from the workspace directory? I'm using kube-prometheus-stack and just overwriting values.yaml.

double-yaya commented 1 year ago

some one resolved this isson?

jeswant commented 1 year ago

I'm having the same issue unfortunately

grafana logger=provisioning.dashboard type=file name=qa-dashboards t=2023-03-27T12:52:16.963445325Z level=error msg="failed to load dashboard from " file=/var/lib/grafana/dashboards/qa-dashboards/general-qa-dashboard.json error=EOF

configMap (grafana-dashboards-qa-dashboards) and cat /var/lib/grafana/dashboards/qa-dashboards/general-qa-dashboard.json are empty

dashboardProviders:
 dashboardproviders.yaml:
   apiVersion: 1
   providers:
   - name: 'default'
     orgId: 1
     folder: ''
     type: file
     disableDeletion: false
     editable: true
     options:
       path: /var/lib/grafana/dashboards/default
   - name: 'qa-dashboards'
     orgId: 1
     folder: 'QA Dashboards'
     # folderUid: qa-dashboards
     type: file
     disableDeletion: false
     editable: true
     options:
       path: /var/lib/grafana/dashboards/qa-dashboards

dashboards:
  default:
    kubernetes-views-pods:
      gnetId: 15760
      datasource: Thanos
  qa-dashboards:
    general-qa-dashboard:
      # folderUid: qa-dashboards
      file: dashboards/general-qa-dashboard.json
      datasource: Thanos

folder structure looks accurate. helm upgrade --install grafana grafana/grafana -n monitoring -f grafana_values.yaml --create-namespace --version "6.52.4"

dminca commented 1 year ago

Have you seen this?

image

https://grafana.com/docs/grafana/latest/administration/provisioning/#provision-folders-structure-from-filesystem-to-grafana

Probably that's why it's not working... I see you're providing a folderUid

jeswant commented 1 year ago

Tested the following scenarios:

Continued to see:

grafana logger=provisioning.dashboard type=file name=qa-dashboards t=2023-03-29T05:25:52.513948565Z level=error msg="failed to load dashboard from " file=/var/lib/graf │ │ ana/dashboards/qa-dashboards/general-qa-dashboard.json error=EOF

Just wondering to see if this feature works atleast for someone deploying via helm charts

jeswant commented 1 year ago

Finally got it working with the following steps and with the values mentioned in my first comment:

helm repo add grafana https://grafana.github.io/helm-charts
helm pull grafana/grafana
helm repo remove grafana 
tar -xvzf grafana-6.52.4.tgz
cp dashboards/general-qa-dashboard.json grafana/dashboards/general-qa-dashboard.json
helm upgrade --install grafana ./grafana --version "6.52.4" -n monitoring -f grafana_values.yaml

HTH!

PrateekKhatri commented 1 year ago

Hi,

Anyone find solution for this, I am also trying to setup dashboards on runtime using helm. Here are my values.yaml file (file name: default-dashboards.yaml):

Screenshot 2023-06-02 at 5 23 39 PM

Folder structure:

Screenshot 2023-06-02 at 5 22 43 PM
abiydv commented 1 year ago

I ran into this issue trying to setup a minimal Grafana instance (with couple of custom dashboards) through ArgoCD. My workaround to deploy dashboards involves the use of kustomize.

At a high level,

I have added the example configs here if it helps anyone.

Paul-Michaud commented 1 year ago

You can use the same method as @abiydv using helm (Thanks for the idea)

values.yaml

grafana:
  # Omited all other variables
  dashboardProviders: 
    dashboardproviders.yaml:
      apiVersion: 1
      providers:
      - name: dashboards
        type: file
        disableDeletion: true
        editable: false
        options:
          path: /var/lib/grafana/dashboards
          foldersFromFilesStructure: true

  dashboardsConfigMaps:
    default: "mydashboards-cm"

# Create a custom variable, we follow the scheme of the dashboards variable from grafana chart
myDashboards:
  default:
    custom-dashboard:
      file: dashboards/custom-dashboard.json

Create a custom template that uses our myDashboards variable, following charts/grafana/templates/dashboards-json-configmap.yaml mydashboards-json-configmap.yaml

{{- if .Values.myDashboards }}
{{ $files := .Files }}
{{- range $provider, $dashboards := .Values.myDashboards }}
apiVersion: v1
kind: ConfigMap
metadata:
  name: mydashboards-cm
  namespace: {{ include "grafana.namespace" $ }}
  labels:
    dashboard-provider: {{ $provider }}
{{- if $dashboards }}
data:
{{- $dashboardFound := false }}
{{- range $key, $value := $dashboards }}
{{- if (or (hasKey $value "json") (hasKey $value "file")) }}
{{- $dashboardFound = true }}
  {{- print $key | nindent 2 }}.json:
    {{- if hasKey $value "json" }}
    |-
      {{- $value.json | nindent 6 }}
    {{- end }}
    {{- if hasKey $value "file" }}
    {{- toYaml ( $files.Get $value.file ) | nindent 4}}
    {{- end }}
{{- end }}
{{- end }}
{{- if not $dashboardFound }}
  {}
{{- end }}
{{- end }}
---
{{- end }}

{{- end }}

The folder looks like this

.
.
├── Chart.lock
├── charts
│   └── grafana-6.0.1.tgz
├── Chart.yaml
├── dashboards
│   └── custom-dashboard.json
├── templates
│   └── mydashboards-json-configmap.yaml
└── values.yaml

And finally, depending on your configuration you should have the file /var/lib/grafana/dashboards/default/custom-dashboard.json in the orange-grafana container of the grafana pod !

dbaltor commented 7 months ago

If you're deploying Grafana using kube-prometheus-stack, the easiest way to install dashboards with your deployment is just creating ConfigMaps with the grafana_dashboard label equals 1 and the data section containing the dashboard json definition as below:

apiVersion: v1
kind: ConfigMap
metadata:
  name: sample-grafana-dashboard
  labels:
     grafana_dashboard: "1"
data:
  k8s-dashboard.json: |-
  [...]

The sidecar is going to pick all your dashboards up as long as they have the correct label. Although you can chuck all dashboards under the same ConfigMap, the team's recommendation is to use one dashboard per ConfigMap. Source: https://github.com/grafana/helm-charts/tree/main/charts/grafana#sidecar-for-dashboards