Use the marketing team's platform to send our emails?

[rfk]

This is a heretical counter-point to #116 which I'm filing for discussion; I'm not sure how I really feel about it in practice, but it will be worth having the discussion either way. It might be that we just close this WONTFIX and refer back to it for future reference. That's why there's a question-mark in the title. But anyway...

The marketing team (is that even/still the right name for them?) use this thing called "SalesForce Marketing Cloud" (previously "ExactTarget") to send emails, and by all accounts they're very good at it. We should try to explicitly compare deliverability metrics between our sends and theirs, but I don't think anyone will be surprised to find that they hit the inbox more than we do.

Rather than trying to copy all the tricks from that platform as in #116, can we switch to sending our emails through the same platform?

The obvious upside would be an increase in successful delivery rate. It would probably also work out cheaper in terms of both $$ and people-hours to consolidate this infrastructure.

But there are also plenty of counter-points to consider, both from the FxA side:

• Security. We use email loops to send security-sensitive verification codes, and every new system we add into the mix changes the shape of the security boundary.
• Complexity. More moving parts means more opportunity for things to go wrong.
• Reliability. We've had issues in the past with the basket API going down and degrading the FxA signup experience. When FxA sends an email we need it sent now, whereas many marketing emails are less time-sensitive.

And the marketing side:

• Would an increase in users entering junk email addresses due to account signups, drag down the overall sender reputation of the system?

Thoughts? /cc @davismtl @mozilla/fxa-devs

[davismtl]

This is an excellent question to ask.

The funny thing here is that I have seen these email systems often kept separate from each other but for the complete opposite reason. Usually marketing has the deliverability problems and you don't want it to negatively impact your transactional emails.

Could there be a turning point in time where our transactional emails become at risk due to marketing? This could potentially happen as marketing ramps up lifecycle initiatives so I'm hesitant about having everything on one platform as a long term solution.

A broader question I ask myself is... regardless of our long-term preferences for sending out emails, should we have the capability to send emails by one or the other as a backup solution? Let me elaborate more... I beleive our email is mission critical. What happens if Gmail starts to put us in spam too tomorrow? Are we equiped to quickly recover from that by moving to another IP or to another ESP?

At the last place I worked, email was our bread and butter. We permanently distributed our emails between two or more email servers. We also had tools to start warming up a new IP within minutes. We got burnt once or twice and felt it had become mission critical to us to have this capability.

Do we believe that email is mission critical to FxA? Is email currently a SPOF? If the answers to these is yes than we have some work ahead of us and perhaps having Exact Target as a our backup would be a wise decision since we already pay for it.

[rfk]

Do we believe that email is mission critical to FxA? Is email currently a SPOF?

Yes, and yes.

[seanmonstar]

Simple questions here: can we get notification of delivery status, or if it bounced etc, and quickly?

[davismtl]

Normally yes. I'll let engineering answer for Amazon specifically.

A great indicator of both of those though is email clickrate. (clicks / emails sent) As soon as that number drastically drops, that's a red flag. We can control tracking on both of those events to calculate the ratio. We track when an email is sent. We can also track when there is a click. At that point, do we re-calculate the rate every 5-15 minutes to determine if something is wrong? (prob depends on email volume)

[rfk]

A great indicator of both of those though is email clickrate. [...] do we re-calculate the rate every 5-15 minutes to determine if something is wrong?

IIUC we currently get this on the same 24-hour turnaround as the other flow metrics in re:dash. We can see the raw counts on a semi-realtime basis in Kibana, but can't do the math to calculate click rate with that tool. And I don't believe this data makes it into datadog in a reliable fashion.

We could use this as a test-case doing more timely dumps into re:dash, and/or consider it a good testcase for other metrics vendors.

can we get notification of delivery status, or if it bounced etc, and quickly?

I'm sure the marketing platform has these metrics, but I don't know how we might go about expatriating them back into the FxA metrics infrastructure.

[seanmonstar]

I apologise for not being clear. I was asking about notifications so that we can use the information to display better errors or other information to a user, not to pipe into metrics.

[rfk]

FWIW I have received some...let's say "frank" feedback from the operations team about the reliability of ExactTarget for the kinds of emails we want to send (i.e. transactional and timely, rather than bulk sends). But the general idea of investigating other platforms is certainly a good one.

[davismtl]

Good to know. I'm feeling like this card is evolving into "investigate new ESPs" and minimize risk of SPOF rather than investigate Exact Target. (also based on discussion in engineering cards about senderscore: https://github.com/mozilla/fxa-auth-mailer/issues/250#issuecomment-277033054 )

@rfk You OK if I close this and create a new feature card?

[rfk]

Yes, I agree, let's close this one out and do an "investigate other ESPs" feature