grafana / mimir

Grafana Mimir provides horizontally scalable, highly available, multi-tenant, long-term storage for Prometheus.
https://grafana.com/oss/mimir/
GNU Affero General Public License v3.0
4.09k stars 524 forks source link

Mimirtool multiple files with same namespace are not allowed #6748

Open andres32168 opened 10 months ago

andres32168 commented 10 months ago

Describe the bug

A clear and concise description of what the bug is.

Mimirtool does not allow to load or sync multiple files within the same namespace For example alerts.yaml and rules.yaml from the mimir-mixin-compiled are not allowed to be deployed to the same namespace: Adding a namespace to alerts.yaml

namespace: mimir
groups:
- name: mimir_alerts
  rules:
  - alert: MimirIngesterUnhealthy
 {...}
namespace: mimir
groups:
- name: mimir_api_1
  rules:
  - expr: histogram_quantile(0.99, sum(rate(cortex_request_duration_seconds_bucket[1m]))
{...}

In my opinion both , rules and alerts, belong to mimir which should be only 1 namespace and not 2.

Of course, it would be possible to have 2 different namespaces for example mimir-rules and mimir-alerts but both belong to mimir and it should be possible to load both into the same namespace.

To Reproduce

Steps to reproduce the behavior:

  1. Add the same namespace to multiple files (see above)
  2. Run following commands:
    ./mimirtool rules sync --rule-dirs=./rules/
    ERRO[0001] repeated namespace attempted to be loaded     file=rules/rules.yaml namespace=mimir
    mimirtool: error: sync operation unsuccessful, unable to parse rules files: file read error, try --help

    or

    ./mimirtool rules load rules/alerts.yaml rules/rules.yaml
    ERRO[0001] repeated namespace attempted to be loaded     file=rules/rules.yaml namespace=mimir
    mimirtool: error: load operation unsuccessful, unable to parse rules files: file read error, try --help

Expected behavior

Multiple files could take the same namespace

Environment

Mimirtool, version 2.10.4 (branch: release-2.10, revision: https://github.com/grafana/mimir/commit/d1f4f1291001db7384ba66ca90b4f753309f262d)

Additional Context

dimitarvdimitrov commented 10 months ago

you can also combine the rule groups from the two files into a single file and feed that into mimirtool.

Mimir itself can only maintain a single set of rule groups per namespace. providing multiple requires mimirtool to do some preprocessing of the files. Whatever that preprocessing is (combining or choosing first/last set of groups to upload), I think it would not be immediately obvious to a user.

andres32168 commented 10 months ago

Yes, we could combine them into a single file and maybe we need to do it. It would be nice to have it in single files because we have a selfservice in our company where every developer writes the alerting rules by his self for his services in multiple files and afterwards we merge them together into a single file. Now we want to introduce recording rules and it would be nice to have 2 files, 1 for alerting and 1 for recording rules.

dark-brains commented 8 months ago

I not see any issue . If you just exprted kube-prmetheus-stack rules and alerts then files have symbolic links. You need to write your command as this: mimirtool rules sync --address=http://localhost:33097 --id=mimir_cluster --rule-dirs=./etc/prometheus/rules/prometheus-prometheus-stack-kube-prom-prometheus-rulefiles-0/..data/

Screenshot from 2024-02-19 22-15-28