[EPIC] Define In App Analytics

russellmorton commented 3 years ago

As a product manager, I want to measure how Taqueria is being used so that I can understand which features are used most frequently as well as how they are used so that I can optimize Taqueria for developers.

USE CASE: Understanding how Taqueria is used will provide us with the necessary information to optimize Taqueria as well as create a roadmap for features that may be the most valuable users. Analytics will also assist in proactively determining any bottlenecks or inefficient processes within Taqueria.

The goal of data collection will be to determine as close to as possible:

Timestamp
Category
Method
Object
Value
Extra data

ACCEPTANCE CRITERIA:

[x] Upon installation users are prompted to explicitly opt-in to usage analytics data collection
[x] Data collected is anonymized
[ ] Data points:
- [x] User ID
- [ ] Session ID
- [x] Taqueria build information
- [x] Data point timestamp
- [x] User action performed
- [ ] All errors
- [ ] Snapshot of plugins used in a project
- [ ] List of operations (taq commands) run in a project (to identify common workflows)
- [ ] System/environment data
- [ ] Version info for dependencies (node, docker, deno, etc)
- [x] Update terms and conditions @neilmangan

ADDITIONAL ITEMS FOR DISCUSSION:

Can we measure specific workflows taken by users? e.g. init - install Ligo - install Taquito - compile etc.?

EXAMPLE METRICS TO BE CALCULATED:

Metric Name Metric Common Short Name Definition How will the metric be used? Comment

Daily Active Users DAU Count of the unique number of users using Taqueria in a day. Trend over time for the number of daily users to understand adoption rate of Taqueria

30 Day Active Users MAU Count of the unique number of users using Taqueria over a rolling 30 day period Trend over time to understand usage over a longer time frame

DAU/MAU Ratio Ratio of daily active users divided by monthly active users Provides us with a proportion of monthly active users who engage with Taqueria in a single day window This is calculated daily using the previous day's DAU divided by MAU. e.g. DMR for 1 December will be (DAU for 30-Nov)/(MAU for 1-Nov-30 Nov)

Daily Installs Unique count of number of installs in a day Provides us with a count of new users in a single day

30 Day Installs Unique count of number of installs in a 30 day period Provides a total number of installs in the preceding 30 days

Install Ratio Daily Installs divided by 30 day Installs Provides us with the proportion of new users that installed Taqueria in the last day. Calculated similarly to DAU/MAU Ratio

Feature usage Most used commands, plugins, etc.

Conversions To be defined based on open question around goal tracking

Metric Name	Metric Common Short Name	Definition	How will the metric be used?	Comment
Daily Active Users	DAU	Count of the unique number of users using Taqueria in a day.	Trend over time for the number of daily users to understand adoption rate of Taqueria
30 Day Active Users	MAU	Count of the unique number of users using Taqueria over a rolling 30 day period	Trend over time to understand usage over a longer time frame
DAU/MAU Ratio		Ratio of daily active users divided by monthly active users	Provides us with a proportion of monthly active users who engage with Taqueria in a single day window	This is calculated daily using the previous day's DAU divided by MAU. e.g. DMR for 1 December will be (DAU for 30-Nov)/(MAU for 1-Nov-30 Nov)
Daily Installs		Unique count of number of installs in a day	Provides us with a count of new users in a single day
30 Day Installs		Unique count of number of installs in a 30 day period	Provides a total number of installs in the preceding 30 days
Install Ratio		Daily Installs divided by 30 day Installs	Provides us with the proportion of new users that installed Taqueria in the last day.	Calculated similarly to DAU/MAU Ratio
Feature usage	Most used commands, plugins, etc.
Conversions	To be defined based on open question around goal tracking