I use grafana -> alertmanager -> signalilo -> icinga2. This way my users don't have to meddle with prometheus/alertmanager yaml files for their alerting needs.
The issue:
It seems display_name gets set from alertname.
However, icinga won't accept certain characters in that field. I think even spaces are invalid. So i have to use simple definitions like "ALERT_01_for_some_service". Which seem to go into service definition (object name, which has restrictions) and display name (which doesn't have a lot of restrictions).
I thought i could pass specific field to signalilo to overwrite this, but i think i cannot.
So, I tried setting icinga_string_display_name with custom display_name and providing a template :
template Service "fix-display-name" {
if (vars.display_name) {
display_name = vars.display_name
}
}
What happens is - signalilo creates an object like this :
object Service "CMB_PVC_K8S_6f0bc2786b5c6528" {
import "generic-service"
(...)
display_name = "CMB_PVC_K8S"
(.....) vars["display_name"] = "Capacity of PVC spool-cc-cmb-test-aggregator-0"
And my custom display_name does not apply.
The problem is that template inheritance happens BEFORE all variables are parsed.
The fix is to import the template at the very end of service definition. I have no way to configure that in signalilo, but such feaure Would Be Nice To Have.
I use grafana -> alertmanager -> signalilo -> icinga2. This way my users don't have to meddle with prometheus/alertmanager yaml files for their alerting needs.
The issue:
It seems display_name gets set from alertname.
However, icinga won't accept certain characters in that field. I think even spaces are invalid. So i have to use simple definitions like "ALERT_01_for_some_service". Which seem to go into service definition (object name, which has restrictions) and display name (which doesn't have a lot of restrictions).
I thought i could pass specific field to signalilo to overwrite this, but i think i cannot.
So, I tried setting icinga_string_display_name with custom display_name and providing a template :
What happens is - signalilo creates an object like this :
And my custom display_name does not apply.
The problem is that template inheritance happens BEFORE all variables are parsed.
The fix is to import the template at the very end of service definition. I have no way to configure that in signalilo, but such feaure Would Be Nice To Have.