Document testing system email delivery #267

Closed treasuretron closed 4 years ago

In GitLab by @richbodo on Aug 29, 2018, 14:55

We have a list of system emails that we send, and their triggers, here:

https://gitlab.com/spacebase/spacebase/wikis/design-specs/email-notification-strategy

[ ] Perform a full regression test on system email delivery as described the above doc, including the things that are not fully implemented. Include the results in a comment on this issue.
[ ] File any bugs encountered.
[ ] If you find anything that is incorrect on the above wiki page, or anything that is not obvious, or run into any problems, enhance the documentation on the above wiki page as needed to make it easier to perform regression tests on system email delivery for the next person.

When testing, note that our prod domain is beta.spacebase.co, and our sandbox is at a URL that mailgun generated. There is a whitelist of recipients in mailgun that is bizarrely located under account prefs when you are logged in there.

See the siteops page to learn more about the current mailgun setup.

SiteOps

In GitLab by @richbodo on Aug 29, 2018, 14:56

changed the description

In GitLab by @richbodo on Aug 30, 2018, 11:56

assigned to @Khiggins42 and @StephenCataldo

In GitLab by @richbodo on Aug 30, 2018, 12:03

@Khiggins42 @StephenCataldo Can you guys estimate this?

In GitLab by @StephenCataldo on Aug 30, 2018, 22:53

Documentation starts here: https://gitlab.com/spacebase/spacebase/wikis/Development/Behat Temp worknotes go here: https://docs.google.com/document/d/1jt-GCTH6bgZN311qyUjHK-soJfcfNLMBoh6rqx8YEpA/edit?usp=sharing

In GitLab by @StephenCataldo on Aug 30, 2018, 22:53

changed time estimate to 1d 4h

In GitLab by @StephenCataldo on Aug 31, 2018, 10:49

mentioned in commit 63f3fc53954fbdef12cd710fe273e335635a95a0

In GitLab by @StephenCataldo on Aug 31, 2018, 10:58

Update on 261: @richbodo @Khiggins42

I'm making pretty fast progress on setting up a very-minimal Behat test for email (sketched in the Temp worknotes file). As discussed there are lots of extra pain points to a Behat test for emails ... for the moment my goal is to get basics and proof of concept, then we can discuss how much finesse. Rich I'd rather avoid meetings till we have more to show and reach decision-points ... but jump in if you think it's important.

Running: lando behat --config=/app/tests/behat.yml --name="emails" works, though you'll have to adjust the email-enabled account in tests/features/emails.feature I'm putting work-notes at the top of emails.feature as I go

Some next steps for either me or Kurt:

Summary messages from Behat ("__ Expect an email after midnight")
Get this running with Drush so it can run on non-lando environments (part of this budget? priority? easy?)
Maybe create some emails within mailgun for testing, that dummy-accounts can use.

In GitLab by @StephenCataldo on Aug 31, 2018, 12:58

mentioned in commit d0ebba0d212402fe6ed7f6c1de871acbe8f2ba0e

In GitLab by @StephenCataldo on Aug 31, 2018, 13:05

mentioned in merge request !214

In GitLab by @StephenCataldo on Aug 31, 2018, 14:15

mentioned in commit a74051d3b2af7b1e8f98aac81d75f9e66df9b08d

In GitLab by @Khiggins42 on Aug 31, 2018, 17:20

mentioned in commit 5d47a309df5a4977a877f4b3caf3ad8bba9a54ac

In GitLab by @StephenCataldo on Sep 5, 2018, 09:58

mentioned in commit dfaf8c347306c3f1daa9c897b503dc8b786376e7

In GitLab by @StephenCataldo on Sep 5, 2018, 13:54

mentioned in commit 3f6726b391e02c7add3c1ce476acfff890e841eb

In GitLab by @StephenCataldo on Sep 7, 2018, 13:50

Time review: About 19 hours (Stephen+Kurt) before two hour respec conversation yesterday (should this be added to estimates). Some simple, hacky email tests work and others close, but instead of sticking with quick click-style effort we're going to explore more clean (and complicated) tests. Two more workdays planned (mostly my time, I don't think Kurt is planning to much, lmk if wrong.)

In GitLab by @StephenCataldo on Sep 7, 2018, 13:50

changed time estimate to 4d

In GitLab by @StephenCataldo on Sep 7, 2018, 13:50

added 2d 3h of time spent at 2018-09-07

In GitLab by @StephenCataldo on Sep 14, 2018, 13:53

This and #263 (overlap, setting up complex testing), and additional 15 hours from last.

In GitLab by @StephenCataldo on Sep 14, 2018, 13:53

added 1d of time spent at 2018-09-14

In GitLab by @StephenCataldo on Sep 14, 2018, 13:55

Meeting time not in last

In GitLab by @StephenCataldo on Sep 14, 2018, 13:55

changed time estimate to 4d 2h

In GitLab by @StephenCataldo on Sep 14, 2018, 13:55

added 2h of time spent at 2018-09-14

In GitLab by @StephenCataldo on Sep 18, 2018, 18:26

marked this issue as related to #264

In GitLab by @StephenCataldo on Sep 20, 2018, 03:12

mentioned in commit c18baffb173c70a0da1d9de0f431b719beeb4b6f

In GitLab by @StephenCataldo on Sep 20, 2018, 03:12

mentioned in commit 381a135b7d6d0a4cd4206e25841db94687141244

In GitLab by @StephenCataldo on Sep 20, 2018, 03:12

mentioned in commit 99c8680126029031df0e849d40ac4ca0c630ff04

In GitLab by @StephenCataldo on Sep 20, 2018, 03:25

Short summary of so far, more details https://docs.google.com/document/d/1jt-GCTH6bgZN311qyUjHK-soJfcfNLMBoh6rqx8YEpA/edit?usp=sharing

1: email_group_administrators_cron, fires after midnight: Test works, but until we fix the group context, it's hand-entered config.
2: New content for site-admins: run just after midnight, ... Works & cleans up well. Now unpublished is newer requirement implemented; also reverse-tested against notifying for published content.
3: New content for org-owners: spacebase_core_cron, run just after midnight, summarizes any actionable unpublished content for org owners. Tests are written and code works for published content. @ToDo, undo that.
4: Users apply to join organizations. When when an admin accepts or rejects ... tests written but @ToDo require javascript test setup to use

In GitLab by @StephenCataldo on Sep 20, 2018, 03:25

added 1d 3h of time spent at 2018-09-20

In GitLab by @richbodo on Oct 22, 2018, 21:27

removed milestone

In GitLab by @richbodo on Dec 11, 2018, 13:51

closed

spacebase / spacebasenz

Document testing system email delivery #267