Email Delivery Issues

[ProbablePrime]

We were getting some reports of email delivery issues.

To resolve this I have a lot of stuff to do:

• [X] Get Microsoft to unblock our IP Addresses on their email side of things
  • This prevents Microsoft email addresses from rejecting our emails
• [x] Working with google to increase our reliability
  • Google has https://postmaster.google.com/ to monitor email delivery to them
  • This also requires some DNS updates
• [x] Resolve OS Ticket having issues with undelivered mail
  • When we fail to send a ticket response a customer, our email server will respond with a delivery status notification.
  • OS Ticket is not setup to handle these correctly.
• [ ] Setup instrumentation to monitor email reliability
  • Much like status.yellowdogman.com I want an alert when our email reliability falls etc.
  • https://github.com/jgosmann/dmarc-metrics-exporter/
  • https://gitlab.hedenstroem.com/go/dmarc-prometheus-exporter
  • [ ] Mail from yellowdogman.com has an incorrect DMARC setup
  • [ ] We're not examining DMARC reports
  • [x] Mail processing for support and moderation was delayed
  • [x] Mail processing for support wasn't actually working.

[ProbablePrime]

Our OS Ticket handling of Undelivered Mail turned out to not be working due to a small capitalization issue in OSTicket's mail parsing. a PR for the fix can be found here: https://github.com/osTicket/osTicket/pull/6663/

A big thank you to: Hikari Akimori, Bredo, Sveken, Jae

and Esa Jokinen on server fault who helped in a question here:https://serverfault.com/questions/1153290/add-original-message-id-to-headers-in-a-postfix-bounce

[oh2fih]

and Esa Jokinen on server fault who helped in a question here:https://serverfault.com/questions/1153290/add-original-message-id-to-headers-in-a-postfix-bounce

I'm glad I could help you find this bug. :sunglasses:

[ProbablePrime]

Today I also found some issues in our incoming mail fetching and processing within our ticket system.

The ticket desk will be faster to respond to emails now and thanks to yesterday's fix it will also flag a failed delivery properly.

References: https://docs.osticket.com/en/latest/Getting%20Started/POP3-IMAP%20Settings.html#schedule-polling

[ProbablePrime]

Metrics are starting to flow on this topic see #1272 for some information. BUT i did discovery a few things:

1. Our email health is pretty damn good now
2. Spam accounts on the wiki need to be handled a little better, they enter fake emails which generate an undeliverable email in our mail systems.
3. Biggest gap right now is DNS records on Yellowdogman.com

[ProbablePrime]

For item 2 on my list, I remember reading a setting or extension on mediawiki that would verify if a domain exists before allowing a user to register.

I can't find this today though oh well.

[shiftyscales]

This sounds promising, @ProbablePrime. https://github.com/quanterium/EmailDNSValidate

[ProbablePrime]

This sounds promising, @ProbablePrime. https://github.com/quanterium/EmailDNSValidate

I installed this but it isn't working at the moment, the code is very simple though so I'll fix it up when I can.

[ProbablePrime]

I re-wrote the extension you found @shiftyscales and it now works, thank you for the help!