Export amplitude events from SalesForce Marketing Cloud into S3

[rfk]

As part of our amplitude integration, we want to correlate marketing email events (such as opens and clicks) with future activity as an FxA user. To do this we'll need to synthesize the events named "mktg - *" from this event taxonomy spreadsheet:

https://docs.google.com/spreadsheets/d/1G_8OJGOxeWXdGJ1Ugmykk33Zsl-qAQL05CONSeD4Uz4/edit#gid=879531055

Rough plan is to:

• Get the SMFC folks to run a periodic job to dump the events into S3
• Have our own periodic job that picks up these dumps, reformats them into the format expected by amplitude, does ID anonymization etc, and puts them in a separate location in S3.
• Pick up the formatted files from S3 and publish them to amplitude via HTTP API.

[rfk]

I filed a bugzilla bug to discuss the mechanics of this from the SFMC side:

https://bugzilla.mozilla.org/show_bug.cgi?id=1409247

[vladikoff]

from mtg: still waiting on SalesForce to export stuff

[vladikoff]

from mtg: still waiting on SalesForce to export stuff

[vladikoff]

from mtg: waiting for an S3 bucket with writing permissions, there is a sample file. need to figure out the pipeline...

[vladikoff]

from mtg: @jrgm need an S3 bucket

[vladikoff]

from mtg: @jrgm taking a look at this today

[vladikoff]

@pmac we have an S3 bucket setup, waiting for data so we can start thinking about how we will process it it for Amplitude

[philbooth]

we can start thinking about how we will process it it for Amplitude

Ref: https://github.com/mozilla/fxa-amplitude-send/pull/38

[pmac]

@pmac we have an S3 bucket setup, waiting for data so we can start thinking about how we will process it it for Amplitude

Great! I'm working with Ben Niolet now on getting him setup with the bucket and he should drop data there soon. Thanks!

[jrgm]

Seems this is instead going to be done with SFTP, yeah? Close this one?

[vladikoff]

@jrgm we should rename

[vladikoff]

from mtg: waiting until end of April for this