Open kristeey opened 8 months ago
Hi @kristeey,
I am guessing that the extra messages are for changes which matched the on_deployment_update
trigger but where the babysitter did nothing because the change was not to the field spec.template.spec.containers[0].image
. In that case, only the mention_enricher did something.
This obviously is not the desired behaviour and we have a fix for it coming soon! (We are moving the filtering functionality to the trigger itself - on_deployment_update - so that it will impact all actions in the playbook.)
Describe the bug Triggers:
on_deployment _update
actions:resource_babysitter
andmention_enricher
sinks: SlackUsing
on_deployment _update
+resource_babysitter
alone works just fine, but when used alongsidemention_enricher
produce multiple (i.e. four per pod) "empty" alerts as well as the actual "configured" alert. The "empty" alerts also triggers whenever there is any change to a deployment, even if the change is not in thefield_to_monitor
defined inresource_babysitter
.I have narrowed it down to being related to the trigger
on_deployment_update
trigger not working well withmention_enricher
action. I have also tested thaton_pod_update
gives the same outcome.on_pod_create
however will only create a single triggering resulting in a single alert.I guess this has something to do with that a single change in a deployment will actually aggregate multiple changes to the deployment manifest. I.e.
annotations.deployment.kubernetes.io/revision
,kubectl.kubernetes.io/last-applied-configuration
,creationTimestamp
,generation
e.t.c. And themention_enricher
action will be triggered for each of these changes instead of combining these changes into a single trigger?Additionally: Alerts will be created even though the change does contain the specific field in specified in the
resource_babysitter
, but in this case we do not get the "configured" alert as expected. Removing theresource_babysitter
action all together while still leaving themention_enricher
action in will give the same result.To Reproduce 1. Configure Robusta helm chart:
Custom Playbook:
Sink:
2. Apply a Dummy deployment to the cluster:
apply to cluster using
kubectl apply -f dummy-deployment.yaml
.nginx:latest
usingkubectl edit deployment dummy-deployment
Expected behavior Get a single alert in the
tmp-robusta-test
slack channel that is enriched with the slack mention@kristian
, when the image of the dummy deployment is changed.Screenshots If applicable, add screenshots to help explain your problem.
Additional context