search

phetsims / perennial

Maintenance tools that won't change with different versions of chipper checked out
MIT License
2 stars 5 forks source link

Issues with build-server email notification #301

Open zepumph opened 1 year ago

zepumph commented 1 year ago

@mattpen noted that @jonathanolson had trouble with getting emails for last weeks maintenance production deploys. I had the same issue with getting emails today and yesterday with ph-scale and ph-scale-basics 1.5.8 and 1.5.8-rc.1.

mattpen commented 1 year ago

I contacted the Email team today via an email request to oithelp@colorado.edu and michael.s.brady@colorado.edu

jonathanolson commented 1 year ago

Emails seemed to give out with yesterday's one also, only got half of the emails (died after natural-selection). But it looked like the deploys completed.

mattpen commented 1 year ago

OIT opened a ticket today:

Your issue INC0944138 has been opened with the Office of Information Technology.

mattpen commented 1 year ago

OIT could not immediately find any problems and requested the message ids of the problematic emails. I went to add logging to successful emails and found that it was already there, so these emails that were not sending were never returning a success or an error in the process. This seems very confusing. I changed the email library we are using to the latest version of nodemailer, which I've been having success with on the website-direct server. I also added message id logging. I verified that these changes were working for a chains build on ox-dev and then deployed the change to production.

mattpen commented 1 year ago

Checked in today. The build server reports that it sent 2 emails to Jesse yesterday, I checked with hiim and he said they were received and he wasn't missing any. Looks like we'll have to check again during a batch release.

mattpen commented 1 year ago

@jonathanolson - have you done any batch releases since Feb 8? If so did you have any issues with receiving build success/failure notifications?