mozilla / fxa-activity-metrics

A server for managing the Firefox Accounts metrics database and pipeline
1 stars 3 forks source link

Analyse which perf timings are most strongly correlated with user behaviour #90

Closed philbooth closed 6 years ago

philbooth commented 6 years ago

In mozilla/fxa-content-server#5608, deployed in train 98 yesterday, we started emitting flow events for raw navtiming data:

fxa=# select distinct type from flow_events
fxa-# where type like 'flow.performance.raw%';
                      type
-------------------------------------------------
 flow.performance.raw.domainLookupEnd
 flow.performance.raw.requestStart
 flow.performance.raw.responseEnd
 flow.performance.raw.domLoading
 flow.performance.raw.unloadEventEnd
 flow.performance.raw.redirectEnd
 flow.performance.raw.domainLookupStart
 flow.performance.raw.responseStart
 flow.performance.raw.domContentLoadedEventEnd
 flow.performance.raw.domComplete
 flow.performance.raw.loadEventEnd
 flow.performance.raw.fetchStart
 flow.performance.raw.domInteractive
 flow.performance.raw.unloadEventStart
 flow.performance.raw.redirectStart
 flow.performance.raw.connectStart
 flow.performance.raw.connectEnd
 flow.performance.raw.secureConnectionStart
 flow.performance.raw.domContentLoadedEventStart
 flow.performance.raw.loadEventStart
(20 rows)

We should analyse which of them is the best predictor of user behaviour.

We should also include flow.performance.auth in the analysis, which is the timing for our custom loaded event on the sign-in and sign-up views. This event is emitted when the app is ready for the user to interact with, so in theory it should be at least as strongly correlated as anything else there (if not more so).

@irrationalagent, I opened this so that we can track your analysis work on the waffleboard and also to let you know that the raw event data has started showing up from today.

vladikoff commented 6 years ago

from mtg: active for train 101 , cc @irrationalagent

vladikoff commented 6 years ago

from mtg: still in the works

vladikoff commented 6 years ago

from mtg: py notebook in the works! waiting for @philbooth to come back to finalize...

vladikoff commented 6 years ago

Notebook here: https://github.com/mozilla/fxa-adhoc-analysis/blob/master/fxa_perf.ipynb