grafana / loki

Like Prometheus, but for logs.
https://grafana.com/loki
GNU Affero General Public License v3.0
23.59k stars 3.41k forks source link

Ability to specify per tenant retention configuration with Helm #8105

Open braunsonm opened 1 year ago

braunsonm commented 1 year ago

Is your feature request related to a problem? Please describe. I would like to be able to specify per tenant overrides when deploying Loki with Helm.

Describe the solution you'd like Values in the helm chart which let you provide contents of an override file that will be mounted at a reliable location (like /etc/loki/overrides.yml)

Describe alternatives you've considered There is no alternative. The chart must support this as you cannot change the mounts to the pod using the helm chart.

Additional context As far as I know this is the only way to specify per tenant retention which means you cannot use this feature if you are deploying with the Helm chart.

hasland commented 6 months ago

Just ran into this and now it's possible. Here's how we achieved it: Chart repo: https://grafana.github.io/helm-charts chart: loki targetRevision: 5.35.0

Create a values.yaml file with:

...
extraObjects:
- apiVersion: v1
    kind: ConfigMap
    metadata:
      name: loki-tenant-override-rules
    data:
      loki-tenant-override-rules.yaml: |-
        overrides:
          "tenant1":
              retention_period: 168h
              retention_stream:
              - selector: '{namespace="prod"}'
                priority: 2
                period: 336h
              - selector: '{container="loki"}'
                priority: 1
                period: 72h
          "tenant2":
              retention_stream:
              - selector: '{container="nginx", level="debug"}'
                priority: 1
                period: 24h

write:
  extraVolumes:
    - name: loki-tenant-override-rules
      configMap:
        name: loki-tenant-override-rules
        items:
          - key: loki-tenant-override-rules.yaml
            path: loki-tenant-override-rules.yaml
        defaultMode: 420
  extraVolumeMounts:
    - name: loki-tenant-override-rules
      mountPath: /etc/loki/config/override

read:
  extraVolumes:
    - name: loki-tenant-override-rules
      configMap:
        name: loki-tenant-override-rules
        items:
          - key: loki-tenant-override-rules.yaml
            path: loki-tenant-override-rules.yaml
        defaultMode: 420
  extraVolumeMounts:
    - name: loki-tenant-override-rules
      mountPath: /etc/loki/config/override

backend:
  extraVolumes:
    - name: loki-tenant-override-rules
      configMap:
        name: loki-tenant-override-rules
        items:
          - key: loki-tenant-override-rules.yaml
            path: loki-tenant-override-rules.yaml
        defaultMode: 420
  extraVolumeMounts:
    - name: loki-tenant-override-rules
      mountPath: /etc/loki/config/override

loki:
  limits_config:
     ...
     retention_period: 320h
     per_tenant_override_config: /etc/loki/config/override/loki-tenant-override-rules.yaml
braunsonm commented 6 months ago

Thanks @hasland ! I think since the time this issue was created they added the ability to have mounts

I'll leave this up because I still would like this to be easier if possible.