Email handoff queue

[molomby]

A mechanism by which keystone apps can add (possibly a significant number of) emails to a DB-based queue. A worker process then generates and hands off the messages in the background.

This builds on the email adapter work in #292. Compared to the adapter-only solution, its..

• More reliable (eg. allows retries)
• More debuggable (ie. logs errors)
• Allows email queuing to be done within a transaction
• Scales better (basically required once you're sending more than a handful of messages)
• Gives us an in-DB log of emails generated

[VinayaSathyanarayana]

Should we consider AMQP (RabbitMQ) in addition to DB based queues?

[molomby]

@VinayaSathyanarayana Quite possibly, yeah. I wouldn't want to force all apps to use AMQP though. I guess we'd need to define a Keystone queue adapter interface, then people could build out a few implementations, including a AMQP adapter. When configuring your email queue you'd be able to pass in any queue adapter instance.

Does that make sense?

[VinayaSathyanarayana]

@molomby Agree with you.

[stale[bot]]

It looks like you haven't had a response in over 3 months. Sorry about that! We've flagged this issue for special attention. It wil be manually reviewed by maintainers, not automatically closed. If you have any additional information please leave us a comment. It really helps! Thank you for you contributions. :)

[MadeByMike]

Should this be part of the core or a package that works with Keystone to achieve this?

[stale[bot]]

It looks like there hasn't been any activity here in over 6 months. Sorry about that! We've flagged this issue for special attention. It wil be manually reviewed by maintainers, not automatically closed. If you have any additional information please leave us a comment. It really helps! Thank you for you contribution. :)

[MadeByMike]

Closing this issue after discussion with Tim. An email queue that works with Keystone is welcome but this can be integrated with a custom GraphQL query. Keystone doesn't need to have an opinion on how email is handled.

[molomby]

Although I agree that "Keystone doesn't need to have an opinion on how email is handled", doing something in this area would have some nice advantages.

What I'm really angling for here is just having a interface defined that would allow functionality that wants to send email (eg. password reset, email address confirmation, notifications, etc.) to be built independently from the actual email gateway integrations and mail queue management.

Total straw-man but imagine we established a pattern like this, without actually implementing what happened on either side:

keystone.queueEmail({
   from: someUser.id,
   to: someOtherUser.id,
   subject: 'Your thing has been updated!',
   body: whateverTemplatingThingImUsing.render(variables)
});

I guess this still doesn't need to be core but there would be great benefit from everyone following some kind of standard.

I just don't want to see one dev build a ks-mailgun-password-reset package and another dev build ks-mandrill-update-notifications package. Instead, the same effort could be spend building ks-mailgun-gateway and ks-mandrill-gateway packages, plus ks-password-reset and ks-update-notifications packages that all work together.

TL:DR; happy to close this for now but we should revisit at some point.