I am trying to install loki-stack in k8s with helm, the pods are running and ready, but I cannot find any logs in grafana web visualization page. kubetctl logs loki-0 -n loki shows the following messages:
level=debug ts=2024-01-25T07:56:55.136072801Z caller=grpc_logging.go:46 method=/frontendv2pb.FrontendForQuerier/QueryResult duration=52.279µs msg="gRPC (success)"
level=error ts=2024-01-25T07:56:55.136151371Z caller=retry.go:73 org_id=fake msg="error processing request" try=0 err="rpc error: code = Code(500) desc = too many unhealthy instances in the ring\n"
...
level=error ts=2024-01-25T07:56:55.136636982Z caller=retry.go:73 org_id=fake msg="error processing request" try=1 err="rpc error: code = Code(500) desc = too many unhealthy instances in the ring\n"
...
level=error ts=2024-01-25T07:56:55.13809032Z caller=stats.go:57 component=frontend msg="failed to record query metrics" err="expected one of the *LokiRequest, *LokiInstantRequest, *LokiSeriesRequest, *LokiLabelNamesRequest, got "
Below is the initial yaml file I use to helm install loki-stack:
I am trying to install loki-stack in k8s with helm, the pods are running and ready, but I cannot find any logs in grafana web visualization page.
kubetctl logs loki-0 -n loki
shows the following messages:Below is the initial yaml file I use to helm install loki-stack:
Below is the config yaml file:
I use the following commands to change config:
Environment: loki version: 2.9(latest) object storage used: baidu object storage
It would be appreciated if anyone can help. Thank you.