Closed irrationalagent closed 3 years ago
User is engage in the qr flow. NB perhaps this flow is sufficiently different from the others that maybe it just needs its own set of events.
FWIW this one feels different to me, and kind of orthogonal. We only have one pairing flow right now, but I can easily imagine a future where the user does a QR-code based pairing flow, but still as to enter their password in order to complete it (say because they need additional encryption keys, or because they haven't authenticated in a very long time). Even today, some users in the QR-code flow may need to enter a TOTP verification code.
from mtg: @irrationalagent Is there anything else to discuss? What train do you think this would land in if so?
User is engage in the qr flow. NB perhaps this flow is sufficiently different from the others that maybe it just needs its own set of events.
FWIW this one feels different to me, and kind of orthogonal. We only have one pairing flow right now, but I can easily imagine a future where the user does a QR-code based pairing flow, but still as to enter their password in order to complete it (say because they need additional encryption keys, or because they haven't authenticated in a very long time). Even today, some users in the QR-code flow may need to enter a TOTP verification code.
At the very least, I'd say we want login_type=qr
for flow_begin and login_complete. But you're right that most other steps in between might be entirely different.
Maybe this just needs someone (probably me, I guess) to deliver a final spec here? Anyone else want to weigh in?
fxa_* - complete
and fxa_* - submit
events based on how we plan to break out specific events for pairing: https://docs.google.com/document/d/1mYtPTswtzawcn-fwVIbJr1N2J-eiLYd6o1ymlB2DnKM/edit#oh look, I'm assigned to this bug! ping me if this is high priority and I'll fit it in soon.
➤ Wil Clouser commented:
We never had a final spec out of this and I'm not sure it's still relevant now that Looker is in play. If there is work to do here, let's revisit with that spec.
The scenerios/flows that a user can go through during the login/signin process are multiplying quickly, much to the benefit of the user (e.g. https://github.com/mozilla/fxa/issues/1756). Rather than playing whack-a-mole by patching our metrics incrementally to take these changes into account, I’d like to propose a couple more systematic modifications that would make things more easily extensible in the future.
These suggestions should all be non-breaking backward-compatible changes, i.e. they should not modify the behavior or meaning of existing events (unless the behavior of the event in question is broken). However, they would change the way we do analyses moving forward. I don’t at all consider this a “final proposal” and appreciate any feedback.
Change 1: New
fxa - flow_begin
amplitude eventtldr: let’s fire a new amplitude event at the beginning of every flow, if possible with an event property that indicates what flow is beginning
This event should fire at the very beginning of every registration and login flow. The purpose of adding this event is
fxa_login - view
event that we currently use), we would be adding to the numerator in our conversion rates without adding to the denominator - in other words, they would become over-inflated.fxa_login - view
orfxa_reg - view
(the QR flow is one such example today). Having a generic “flow starts here” event for all flows will allow us to change the first steps in the login/reg funnels without worrying about breaking conversion rates.The good news is that we already have a
flow.begin
event in our re:dash metrics, so the solution may be a simple as mapping this to an amplitude event. But before doing so we should consider:flow.begin
).flow.begin
currently fires in more places than just the beginning of the login and reg flows, e.g. when a user opens prefs, etc. It also fires on RP top of funnel eventsfxa_email_first - view
and the newfxa_rp_button - view
. Maybe that’s OKflow_type
. The tricky thing here is we don’t always know the ultimate “destination flow” at the time thatflow.begin
fires - e.g. when the start of the flow is email-first. Maybe that’s would be OK and we just have an amplitude event propertyflow_type
that could belogin
,reg
,email-first
,prefs
,qr
,button
etc. This event property would allow easy segmentation of users starting particular flows.Change 2: add a new event property
login_type
to login amplitude eventstldr we should add a new event property to all fxa_login events that denotes which version of the login flow a user is in
Without adding or removing existing events in the login/signin flow, we could add an event property to these events that denotes the type of login flow the user is engaged in. For example, we could have the property denote the most demanding action that the user needs to take to complete the flow. See the chart below for an example, ordered in descending order of user demand:
qr
confirm
password
click
auto
As noted above, one wrinkle here is that we sometimes do not “know” which version of the login flow a user will get until they complete an initial step, e.g. email-first. This means that we cannot attach this event property (in all cases) to the
fxa - flow_begin
event proposed above. Thus, we should (1) always attach it to the first event of the categoryfxa_login - *
- (usually this will meanfxa_login - view
), as well as (2) tofxa - flow_begin
where possible.All told, these two changes will allow us to
Thoughts? CC @davismtl
┆Issue is synchronized with this Jira Task ┆Issue Number: FXA-333