There was a file permission error that recently prevented around 80 emails in the job queue from being processed and it wasn't noticed for about a week.
Proposal:
[x] Develop a shell script to check for jobs that have been delayed longer than expected (e.g. an hour)
[x] Send notifications to the #webdev channel in cber.slack.com if a job meets this criteria
[x] Run this script in an hourly cron job
[x] Edit records for all of the jobs that were just run (2017-08-29) to make it seem like they were completed promptly (so the "average job existence" stats on the queue monitoring page aren't ruined)
There was a file permission error that recently prevented around 80 emails in the job queue from being processed and it wasn't noticed for about a week.
Proposal: