not able to authenticate in ses

[dauntlessnomad]

What did you do? Configured ses to send alert emails.

What did you expect to see? emails receiving my inbox. What did you see instead? Under which circumstances?

Oct  8 09:51:02 ip-10-64-64-250 alertmanager[34700]: level=error ts=2020-10-08T09:51:02.908Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="email/email[0]: notify retry canceled after 5 attempts: *smtp.plainAuth auth: 535 Authentication Credentials Invalid"
Oct  8 09:51:02 ip-10-64-64-250 alertmanager[34700]: level=error ts=2020-10-08T09:51:02.922Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="email/email[0]: notify retry canceled after 5 attempts: *smtp.plainAuth auth: 535 Authentication Credentials Invalid"
Oct  8 09:51:02 ip-10-64-64-250 alertmanager[34700]: level=error ts=2020-10-08T09:51:02.983Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="email/email[0]: notify retry canceled after 5 attempts: *smtp.plainAuth auth: 535 Authentication Credentials Invalid"
Oct  8 09:51:03 ip-10-64-64-250 alertmanager[34700]: level=error ts=2020-10-08T09:51:03.003Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=12 err="email/email[0]: notify retry canceled after 5 attempts: *smtp.plainAuth auth: 535 Authentication Credentials Invalid"
Oct  8 09:51:03 ip-10-64-64-250 alertmanager[34700]: level=warn ts=2020-10-08T09:51:03.756Z caller=notify.go:674 component=dispatcher receiver=email integration=email[0] msg="Notify attempt failed, will retry later" attempts=1 err="*smtp.plainAuth auth: 535 Authentication Credentials Invalid"

Environment
production AWS

• System information: Centos 8. Linux 4.18.0-193.19.1.el8_2.x86_64 x86_64

• Alertmanager version: 0.21.0

• Prometheus version: v2.21.0.1

• Alertmanager configuration file:

  
  global:
  resolve_timeout: 5m

route: group_by: ['alertname','instance', 'alert'] group_wait: 10s group_interval: 10s repeat_interval: 1h receiver: 'email' receivers:

• name: email email_configs:
  • to: 'x@x.com' from: 'noreply@x.com' smarthost: 'email-smtp.us-east-1.amazonaws.com:587' auth_username: '' auth_password: '' require_tls: yes

inhibit_rules:

• source_match: severity: 'critical' target_match: severity: 'warning' equal: ['alertname', 'dev', 'instance']

• Logs:

  Oct  8 09:51:02 ip-10-64-64-250 alertmanager[34700]: level=error ts=2020-10-08T09:51:02.908Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="email/email[0]: notify retry canceled after 5 attempts: *smtp.plainAuth auth: 535 Authentication Credentials Invalid"
  Oct  8 09:51:02 ip-10-64-64-250 alertmanager[34700]: level=error ts=2020-10-08T09:51:02.922Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="email/email[0]: notify retry canceled after 5 attempts: *smtp.plainAuth auth: 535 Authentication Credentials Invalid"
  Oct  8 09:51:02 ip-10-64-64-250 alertmanager[34700]: level=error ts=2020-10-08T09:51:02.983Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="email/email[0]: notify retry canceled after 5 attempts: *smtp.plainAuth auth: 535 Authentication Credentials Invalid"
  Oct  8 09:51:03 ip-10-64-64-250 alertmanager[34700]: level=error ts=2020-10-08T09:51:03.003Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=12 err="email/email[0]: notify retry canceled after 5 attempts: *smtp.plainAuth auth: 535 Authentication Credentials Invalid"
  Oct  8 09:51:03 ip-10-64-64-250 alertmanager[34700]: level=warn ts=2020-10-08T09:51:03.756Z caller=notify.go:674 component=dispatcher receiver=email integration=email[0] msg="Notify attempt failed, will retry later" attempts=1 err="*smtp.plainAuth auth: 535 Authentication Credentials Invalid" 

[simonpasquier]

535 Authentication Credentials Invalid indicates that the server rejects the authentication. You need to double check the credentials and maybe ask the SES support.

[dionisiemoscalu]

HI @tomahawk-pilot ,

Are you doing it via helm ? Thanks,

[dionisiemoscalu]

@simonpasquier is not related to the SES services, i have same problem

The workaround is to add password throw Alertmanger in Grafana UI (but is not right solution)