search

canonical / kafka-operator

Kafka VM operator
Apache License 2.0
6 stars 12 forks source link

Not Replicated Topic alerts has no topic name in the labels #160

Closed err404r closed 7 months ago

err404r commented 9 months ago

Charm is exporting several alert rules to prometheus, and one of them is the rule to monitore not replicated topics. This alerts have a tendency to trigger randomly during working, and after some discussions with the customer we want to silence it per topic based. But this is not possible, because topic name is not exported into labels list.

Also maybe we should just increase duration before alert is triggered because mostly we observe this alert on temporary bases.

github-actions[bot] commented 9 months ago

https://warthogs.atlassian.net/browse/DPE-3213

alexismosquera commented 8 months ago

Hi team. What is the current status of this?

deusebio commented 7 months ago

We are currently investigating this, as the alert would seem to have this information. We are going to try this out on a test (under-replicated) environment, indeed on kafka-k8s operator (which has the same alerting rule for Not Replicated Topic) this information is present

Screenshot from 2024-02-23 11-45-57

deusebio commented 7 months ago

Yes, I can confirm that also on kafka for vm/iaas environment (revision 153) has the name of the topic and the partition number. I'm going to close the issue. Please let us know if you are still having some issues with this. Feel free to reach out to the Data Platform room in matrix, to get support.