grafana / helm-charts

Apache License 2.0
1.62k stars 2.25k forks source link

Bug with loki-stack chart while adding datasource #3149

Open dk03051996 opened 4 months ago

dk03051996 commented 4 months ago

Hi, I am creating this issue to let you know that with recent version, I tried adding datasource in separate grafana as http://loki:3100 and try to save and test but it fails with error "Unable to connect with Loki. Please check the server logs for more details." but I am able to see logs in dashboard. This is happening in GKE and kind cluster both as testing environment. Please take a look as so many can get confused with this.

Getting below error in grafana pods "logger=tsdb.loki endpoint=checkHealth pluginId=loki dsName=loki dsUID=cdn1r5hikan7kc uname=admin fromAlert=false t=2024-05-30T11:24:42.682252191Z level=info msg="Prepared request to Loki" duration=23.256µs queriesLength=1 stage=prepareRequest runInParallel=false logger=tsdb.loki endpoint=checkHealth pluginId=loki dsName=loki dsUID=cdn1r5hikan7kc uname=admin fromAlert=false t=2024-05-30T11:24:42.861757897Z level=error msg="Error received from Loki" duration=179.38691ms stage=databaseRequest statusCode=400 contentLength=65 start=1970-01-01T00:00:01Z end=1970-01-01T00:00:04Z step=1s query=vector(1)+vector(1) queryType=instant direction=backward maxLines=0 supportingQueryType=none lokiHost=loki.monitoring.svc.cluster.local:3100 lokiPath=/loki/api/v1/query status=error error="parse error at line 1, col 1: syntax error: unexpected IDENTIFIER" statusSource=downstream logger=tsdb.loki endpoint=CheckHealth t=2024-05-30T11:24:42.861870966Z level=error msg="Loki health check failed" error="error from loki: parse error at line 1, col 1: syntax error: unexpected IDENTIFIER" logger=context userId=1 orgId=1 uname=admin t=2024-05-30T11:24:42.861963433Z level=info msg="Request Completed" method=GET path=/api/datasources/uid/cdn1r5hikan7kc/health status=400 remote_addr=127.0.0.1 time_ms=181 duration=181.02212ms size=106 referer=http://localhost:3001/connections/datasources/edit/cdn1r5hikan7kc handler=/api/datasources/uid/:uid/health status_source=server"

spuyet commented 4 months ago

I just got the same issue, the endpoint returns a parser error while we can still send query to the data source.

dk03051996 commented 4 months ago

looks like loki-stack is not being maintained properly.

Sheikh-Abubaker commented 3 months ago

loki-stack is deprecated

EHJ-52n commented 3 weeks ago

@Sheikh-Abubaker Is there any successor to loki-stack? If not, does anyone know a good and helm driven way to add loki to my kube-prometheus-stack?

Sheikh-Abubaker commented 3 weeks ago

@EHJ-52n try using loki

erkules commented 3 weeks ago

but then the Chart should be updated with deprecated: true

lavahot commented 2 weeks ago

loki-stack is deprecated

Will there be migration documentation to move from loki-stack?