Loki is crashed after having storage capacity full.

[neerajsinghnegi]

Describe the bug We deployed the Loki pod in our production cluster, but after some time, its storage became full, causing the Loki pod to crash. As a result, we missed the logs for that time period. Currently, we manually delete data as needed. Additionally, we have configured a retention period of 12 days.

To Reproduce Steps to reproduce the behavior:

1. Use image: grafana/loki:2.6.1 to run loki pod in k8s.
2. And check if we have logs more than retention period.

Expected behavior Pod Crashes after some time.

Environment:

• Infrastructure: Kubernetes
• Deployment tool: helm chat version: loki-stack-2.10.2 app version: v2.9.3