grafana / grafana-operator

An operator for Grafana that installs and manages Grafana instances, Dashboards and Datasources through Kubernetes/OpenShift CRs
https://grafana.github.io/grafana-operator/
Apache License 2.0
863 stars 384 forks source link

[Bug] Unable to deploy Grafana with Persistent Volume on OpenShift 4.15 #1606

Closed duyphuong82 closed 2 months ago

duyphuong82 commented 2 months ago

Log from pod: logger=settings t=2024-07-08T07:46:20.97664063Z level=info msg="Starting Grafana" version=9.4.3 commit=cf0a135595 branch=HEAD compiled=2023-03-02T20:28:42Z logger=settings t=2024-07-08T07:46:20.976857002Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." logger=settings t=2024-07-08T07:46:20.976873472Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini logger=settings t=2024-07-08T07:46:20.976878666Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini logger=settings t=2024-07-08T07:46:20.976882473Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" logger=settings t=2024-07-08T07:46:20.9768856Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" logger=settings t=2024-07-08T07:46:20.976888497Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" logger=settings t=2024-07-08T07:46:20.976891435Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" logger=settings t=2024-07-08T07:46:20.976894387Z level=info msg="Config overridden from command line" arg="default.log.mode=console" logger=settings t=2024-07-08T07:46:20.976897674Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" logger=settings t=2024-07-08T07:46:20.976900696Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" logger=settings t=2024-07-08T07:46:20.976903384Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" logger=settings t=2024-07-08T07:46:20.976906044Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" logger=settings t=2024-07-08T07:46:20.976908758Z level=info msg="Config overridden from Environment variable" var="GF_SECURITY_ADMIN_USER=root" logger=settings t=2024-07-08T07:46:20.976911974Z level=info msg="Config overridden from Environment variable" var="GF_SECURITY_ADMIN_PASSWORD=*****" logger=settings t=2024-07-08T07:46:20.976915663Z level=info msg="Path Home" path=/usr/share/grafana logger=settings t=2024-07-08T07:46:20.976918513Z level=info msg="Path Data" path=/var/lib/grafana logger=settings t=2024-07-08T07:46:20.976921271Z level=info msg="Path Logs" path=/var/log/grafana logger=settings t=2024-07-08T07:46:20.976924272Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins logger=settings t=2024-07-08T07:46:20.976927277Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning logger=settings t=2024-07-08T07:46:20.976931748Z level=info msg="App mode production" logger=sqlstore t=2024-07-08T07:46:20.976979205Z level=info msg="Connecting to DB" dbtype=sqlite3

#######################################

Deployment:

kind: Grafana apiVersion: grafana.integreatly.org/v1beta1 metadata: labels: dashboards: grafana folders: grafana name: grafana namespace: monitoring spec: persistentVolumeClaim: spec: accessModes:

https://grafana.github.io/grafana-operator/docs/examples/persistent_volume/readme/

theSuess commented 2 months ago

From your initial report, I see that you're using NFS as your backing storage. NFS volumes in OpenShift are tricky as OpenShift pods run with restricted privileges and non-root UIDs. This means you have to match the GID/UID with the values of the NFS export.

For more information, see the OpenShift documentation on this topic: https://docs.openshift.com/container-platform/4.16/storage/persistent_storage/persistent-storage-nfs.html#nfs-volume-security_persistent-storage-nfs

duyphuong82 commented 2 months ago

Hello @theSuess So, what are the best practices for deploying Grafana with persistent volume on OpenShift? Can you please help?

theSuess commented 2 months ago

You need to ensure the permissions between the NFS share and the deployment match. This is not something we can give guidance on, as every NFS setup will be different. The prefered method to use persistent volumes is using CSI drivers. When using those, OpenShift will take care of the permission management for you.

If you decide to stick with NFS, you can use the deployment overrides to supply supplemental groups and modify the security context or match the namespace GID on your NFS server. This is a general OpenShift issue, not operator specific.

duyphuong82 commented 2 months ago

Thanks so much, @theSuess I'll try again to suggest from you.