Notification "trigger 'on-rollout-step-completed' is not configured" error during rollout

[mdfaizsiddiqui]

Checklist:

• [x] I've included steps to reproduce the bug.
• [x] I've included the version of argo rollouts.

Describe the bug

I am seeing below error during rollouts of all the applications I have but able to receive notification on slack. I also checked the configuration, all the triggers and templates are setup correctly. One thing to know here in my setup argo rollout exists in argo-rollouts namespace whereas the applications exists in a different namespace.

Notifications failed to send for eventReason RolloutStepCompleted with error: trigger 'on-rollout-step-completed' is not configured

To Reproduce

• I've deployed argo-rollout via the official helm chart, this is the values.yaml file -

---
notifications:
  secret:
    create: true

  notifiers:
    service.slack: |
      token: $slack-token

  triggers:
    trigger.on-analysis-run-error: |
      - send: [analysis-run-error]
    trigger.on-analysis-run-failed: |
      - send: [analysis-run-failed]
    trigger.on-analysis-run-running: |
      - send: [analysis-run-running]
    trigger.on-rollout-aborted: |
      - send: [rollout-aborted]
    trigger.on-rollout-completed: |
      - send: [rollout-completed]
    trigger.on-rollout-paused: |
      - send: [rollout-paused]
    trigger.on-rollout-step-completed: |
      - send: [rollout-step-completed]
    trigger.on-rollout-updated: |
      - send: [rollout-updated]
    trigger.on-scaling-replica-set: |
      - send: [scaling-replicaset]

  templates:
    template.analysis-run-error: |
      message: Rollout {{.rollout.metadata.name}}'s analysis run is in error state.
      email:
        subject: Rollout {{.rollout.metadata.name}}'s analysis run is in error state.
      slack:
        attachments: |
            [{
              "title": "{{ .rollout.metadata.name}}",
              "color": "#ECB22E",
              "fields": [
              {
                "title": "Strategy",
                "value": "{{if .rollout.spec.strategy.blueGreen}}BlueGreen{{end}}{{if .rollout.spec.strategy.canary}}Canary{{end}}",
                "short": true
              }
              {{range $index, $c := .rollout.spec.template.spec.containers}}
                {{if not $index}},{{end}}
                {{if $index}},{{end}}
                {
                  "title": "{{$c.name}}",
                  "value": "{{$c.image}}",
                  "short": true
                }
              {{end}}
              ]
            }]
    template.analysis-run-failed: |
      message: Rollout {{.rollout.metadata.name}}'s analysis run failed.
      email:
        subject: Rollout {{.rollout.metadata.name}}'s analysis run failed.
      slack:
        attachments: |
            [{
              "title": "{{ .rollout.metadata.name}}",
              "color": "#E01E5A",
              "fields": [
              {
                "title": "Strategy",
                "value": "{{if .rollout.spec.strategy.blueGreen}}BlueGreen{{end}}{{if .rollout.spec.strategy.canary}}Canary{{end}}",
                "short": true
              }
              {{range $index, $c := .rollout.spec.template.spec.containers}}
                {{if not $index}},{{end}}
                {{if $index}},{{end}}
                {
                  "title": "{{$c.name}}",
                  "value": "{{$c.image}}",
                  "short": true
                }
              {{end}}
              ]
            }]
    template.analysis-run-running: |
      message: Rollout {{.rollout.metadata.name}}'s analysis run is running.
      email:
        subject: Rollout {{.rollout.metadata.name}}'s analysis run is running.
      slack:
        attachments: |
            [{
              "title": "{{ .rollout.metadata.name}}",
              "color": "#18be52",
              "fields": [
              {
                "title": "Strategy",
                "value": "{{if .rollout.spec.strategy.blueGreen}}BlueGreen{{end}}{{if .rollout.spec.strategy.canary}}Canary{{end}}",
                "short": true
              }
              {{range $index, $c := .rollout.spec.template.spec.containers}}
                {{if not $index}},{{end}}
                {{if $index}},{{end}}
                {
                  "title": "{{$c.name}}",
                  "value": "{{$c.image}}",
                  "short": true
                }
              {{end}}
              ]
            }]
    template.rollout-aborted: |
      message: Rollout {{.rollout.metadata.name}} has been aborted.
      email:
        subject: Rollout {{.rollout.metadata.name}} has been aborted.
      slack:
        attachments: |
            [{
              "title": "{{ .rollout.metadata.name}}",
              "color": "#E01E5A",
              "fields": [
              {
                "title": "Strategy",
                "value": "{{if .rollout.spec.strategy.blueGreen}}BlueGreen{{end}}{{if .rollout.spec.strategy.canary}}Canary{{end}}",
                "short": true
              }
              {{range $index, $c := .rollout.spec.template.spec.containers}}
                {{if not $index}},{{end}}
                {{if $index}},{{end}}
                {
                  "title": "{{$c.name}}",
                  "value": "{{$c.image}}",
                  "short": true
                }
              {{end}}
              ]
            }]
    template.rollout-completed: |
      message: Rollout {{.rollout.metadata.name}} has been completed.
      email:
        subject: Rollout {{.rollout.metadata.name}} has been completed.
      slack:
        attachments: |
            [{
              "title": "{{ .rollout.metadata.name}}",
              "color": "#18be52",
              "fields": [
              {
                "title": "Strategy",
                "value": "{{if .rollout.spec.strategy.blueGreen}}BlueGreen{{end}}{{if .rollout.spec.strategy.canary}}Canary{{end}}",
                "short": true
              }
              {{range $index, $c := .rollout.spec.template.spec.containers}}
                {{if not $index}},{{end}}
                {{if $index}},{{end}}
                {
                  "title": "{{$c.name}}",
                  "value": "{{$c.image}}",
                  "short": true
                }
              {{end}}
              ]
            }]
    template.rollout-paused: |
      message: Rollout {{.rollout.metadata.name}} has been paused.
      email:
        subject: Rollout {{.rollout.metadata.name}} has been paused.
      slack:
        attachments: |
            [{
              "title": "{{ .rollout.metadata.name}}",
              "color": "#18be52",
              "fields": [
              {
                "title": "Strategy",
                "value": "{{if .rollout.spec.strategy.blueGreen}}BlueGreen{{end}}{{if .rollout.spec.strategy.canary}}Canary{{end}}",
                "short": true
              }
              {{range $index, $c := .rollout.spec.template.spec.containers}}
                {{if not $index}},{{end}}
                {{if $index}},{{end}}
                {
                  "title": "{{$c.name}}",
                  "value": "{{$c.image}}",
                  "short": true
                }
              {{end}}
              ]
            }]
    template.rollout-step-completed: |
      message: Rollout {{.rollout.metadata.name}} step number {{ add .rollout.status.currentStepIndex 1}}/{{len .rollout.spec.strategy.canary.steps}} has been completed.
      email:
        subject: Rollout {{.rollout.metadata.name}} step number {{ add .rollout.status.currentStepIndex 1}}/{{len .rollout.spec.strategy.canary.steps}} has been completed.
      slack:
        attachments: |
            [{
              "title": "{{ .rollout.metadata.name}}",
              "color": "#18be52",
              "fields": [
              {
                "title": "Strategy",
                "value": "{{if .rollout.spec.strategy.blueGreen}}BlueGreen{{end}}{{if .rollout.spec.strategy.canary}}Canary{{end}}",
                "short": true
              },
              {
                "title": "Step completed",
                "value": "{{add .rollout.status.currentStepIndex 1}}/{{len .rollout.spec.strategy.canary.steps}}",
                "short": true
              }
              {{range $index, $c := .rollout.spec.template.spec.containers}}
                {{if not $index}},{{end}}
                {{if $index}},{{end}}
                {
                  "title": "{{$c.name}}",
                  "value": "{{$c.image}}",
                  "short": true
                }
              {{end}}
              ]
            }]
    template.rollout-updated: |
      message: Rollout {{.rollout.metadata.name}} has been updated.
      email:
        subject: Rollout {{.rollout.metadata.name}} has been updated.
      slack:
        attachments: |
            [{
              "title": "{{ .rollout.metadata.name}}",
              "color": "#18be52",
              "fields": [
              {
                "title": "Strategy",
                "value": "{{if .rollout.spec.strategy.blueGreen}}BlueGreen{{end}}{{if .rollout.spec.strategy.canary}}Canary{{end}}",
                "short": true
              }
              {{range $index, $c := .rollout.spec.template.spec.containers}}
                {{if not $index}},{{end}}
                {{if $index}},{{end}}
                {
                  "title": "{{$c.name}}",
                  "value": "{{$c.image}}",
                  "short": true
                }
              {{end}}
              ]
            }]
    template.scaling-replicaset: |
      message: Scaling Rollout {{.rollout.metadata.name}}'s replicaset to {{.rollout.spec.replicas}}.
      email:
        subject: Scaling Rollout {{.rollout.metadata.name}}'s replcaset to {{.rollout.spec.replicas}}.
      slack:
        attachments: |
            [{
              "title": "{{ .rollout.metadata.name}}",
              "color": "#18be52",
              "fields": [
              {
                "title": "Strategy",
                "value": "{{if .rollout.spec.strategy.blueGreen}}BlueGreen{{end}}{{if .rollout.spec.strategy.canary}}Canary{{end}}",
                "short": true
              },
              {
                "title": "Desired replica",
                "value": "{{.rollout.spec.replicas}}",
                "short": true
              },
              {
                "title": "Updated replicas",
                "value": "{{.rollout.status.updatedReplicas}}",
                "short": true
              }
              {{range $index, $c := .rollout.spec.template.spec.containers}}
                {{if not $index}},{{end}}
                {{if $index}},{{end}}
                {
                  "title": "{{$c.name}}",
                  "value": "{{$c.image}}",
                  "short": true
                }
              {{end}}
              ]
            }]

With the above values file I am able to see notification configmap and secrets generated perfectly fine -

kubectl get configmap -n argo-rollouts | grep notification
argo-rollouts-notification-configmap   19     88d

kubectl get cm argo-rollouts-notification-configmap -n argo-rollouts -oyaml | grep on-rollout-step-completed -A5
  trigger.on-rollout-step-completed: |
    - send: [rollout-step-completed]

kubectl get secret -n argo-rollouts | grep notification
argo-rollouts-notification-secret   Opaque   1      88d

• My rollout object looks something like this -

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  annotations:
    notifications.argoproj.io/subscribe.on-rollout-aborted.slack: argo-events
    notifications.argoproj.io/subscribe.on-rollout-step-completed.slack: argo-events
...

Expected behavior

There should not be any error in the logs as we already have the required configuration and also the notification is working fine.

Screenshots

Version

1.6.0

Logs

# Paste the logs from the rollout controller

# Logs for the entire controller:
kubectl logs -n argo-rollouts deployment/argo-rollouts

# Logs for a specific rollout:
kubectl logs -n argo-rollouts deployment/argo-rollouts | grep rollout=<ROLLOUTNAME

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.

[mohamedragab2024]

Any updates on this issue , we still facing the same issue . Notifications failed to send for eventReason RolloutPaused with error: [trigger 'on-rollout-paused' is not configured]