Closed frzifus closed 2 weeks ago
By default OpenShift does not allow writing the root directory. Thats why the data dirs for grafana, prometheus and loki can not be created and the pod never becomes ready.
By default OpenShift does not allow writing the root directory. Thats why the data dirs for grafana, prometheus and loki can not be created and the pod never becomes ready.
Logs
grafana
``` logger=featuremgmt t=2024-10-16T09:08:46.588576128Z level=info msg=FeatureToggles cloudWatchNewLabelParsing=true cloudWatchCrossAccountQuerying=true logsExploreTableVisualisation=true logRowsPopoverMenu=true lokiMetricDataplane=true exploreMetrics=true publicDashboards=true alertingNoDataErrorExecution=true addFieldFromCalculationStatFunctions=true tlsMemcached=true recordedQueriesMulti=true prometheusAzureOverrideAudience=true prometheusConfigOverhaulAuth=true influxdbBackendMigration=true lokiQueryHints=true panelMonitoring=true lokiStructuredMetadata=true kubernetesPlaylists=true managedPluginsInstall=true prometheusDataplane=true transformationsVariableSupport=true topnav=true cloudWatchRoundUpEndTime=true correlations=true alertingSimplifiedRouting=true recoveryThreshold=true angularDeprecationUI=true ssoSettingsApi=true prometheusMetricEncyclopedia=true alertingInsights=true nestedFolders=true autoMigrateXYChartPanel=true awsAsyncQueryCaching=true dataplaneFrontendFallback=true logsContextDatasourceUi=true transformationsRedesign=true lokiQuerySplitting=true annotationPermissionUpdate=true groupToNestedTableTransformation=true logsInfiniteScrolling=true dashgpt=true formatString=true Error: ✗ failed to connect to database: mkdir /otel-lgtm/grafana/data: permission denied ```Loki
``` mkdir /loki: permission denied error initialising module: ruler-storage github.com/grafana/dskit/modules.(*Manager).initModule /src/loki/vendor/github.com/grafana/dskit/modules/modules.go:138 github.com/grafana/dskit/modules.(*Manager).InitModuleServices /src/loki/vendor/github.com/grafana/dskit/modules/modules.go:108 github.com/grafana/loki/v3/pkg/loki.(*Loki).Run /src/loki/pkg/loki/loki.go:490 main.main /src/loki/cmd/loki/main.go:129 runtime.main /usr/local/go/src/runtime/proc.go:271 runtime.goexit /usr/local/go/src/runtime/asm_amd64.s:1695 level=info ts=2024-10-16T09:10:33.236996873Z caller=main.go:126 msg="Starting Loki" version="(version=3.2.0, branch=k218, revision=659f5421)" level=info ts=2024-10-16T09:10:33.237271697Z caller=main.go:127 msg="Loading configuration file" filename=./loki-config.yaml ```prometheus
``` ts=2024-10-16T09:28:53.459Z caller=query_logger.go:114 level=error component=activeQueryTracker msg="Error opening query log file" file=/data/prometheus/queries.active err="open /data/prometheus/queries.active: no such file or directory" panic: Unable to create mmap-ed active query log goroutine 1 [running]: github.com/prometheus/prometheus/promql.NewActiveQueryTracker({0x7fffe0fa2a69, 0x10}, 0x14, {0x451ec40, 0xc000739090}) /app/promql/query_logger.go:146 +0x425 main.main() /app/cmd/prometheus/main.go:776 +0x891e sh-5.1$ mkdir /data/prometheus mkdir: cannot create directory '/data/prometheus': No such file or directory ```