alphagov / govuk-prototype-kit-private-beta

⚠️ This repository has been archived and the changes have been merged into https://github.com/alphagov/govuk-prototype-kit
MIT License
1 stars 2 forks source link

Track app usage #25

Closed joelanman closed 6 years ago

joelanman commented 6 years ago

UPDATE

I had misunderstood custom dimensions, and we should use them to track operating system, node version, prototype kit version

https://docs.google.com/document/d/13or0ac7H8TdfWfMSt8xnsGgOuTV8jp7CuEtPa6qcP9U/edit#


This work adds opt-in tracking of app usage, using Google Analytics.

It asks for user's permission when first run.

The app then records the user's answer in usage-data-config.js.

If the answer was yes:

It also records a unique user ID in the same file.

It sends an Event to Google Analytics, with:

I've refactored start.js slightly into a series of functions, this makes it a lot easier to handle the correct flow, based on whether the user is opted in or out.

Known issues

Screens

image

image

When recorded in GA, the event looks like this:

image

joelanman commented 6 years ago

@hannalaakso The data recorded is documented here: https://github.com/alphagov/govuk-prototype-kit-private-beta/blob/d8b51e8a92fce5a231370ddf1c9b6f0f995c385e/docs/documentation/usage-data.md

Nice point about reminding people every start, I'll look into that.

joelanman commented 6 years ago

On reviewing this with @alex-ju, I'm not sure the Google Analytics approach can work.

We want to know for example, what % of our users are on Windows. GA does this with Dimensions, which I don't think we can use (custom dimensions are managed by hand, and only have 200 different values).

GA Events (the approach taken in this PR) are not reported on as above. They are reported on as Total events, or Unique events. Either one one show a user starting the kit twice on Windows to have 2 events, and a user starting once on OSX to have 1. This does not give us the data we need.

NickColley commented 6 years ago

Would this stop heroku from deploying the prototype kit since it could get stuck on this question?

joelanman commented 6 years ago

@nickcolley it's a good question, but Heroku doesn't run npm start - it runs procfile instead

alex-ju commented 6 years ago

Updated with using custom dimensions for storing persistent data (Operating System, Prototype Kit Version and Node Version).

alex-ju commented 6 years ago

Updated with using custom dimensions for storing persistent data (Operating System, Prototype Kit Version and Node Version).

Refactored the code. @nickcolley can you take a look when you have time (no pressure)?

alex-ju commented 6 years ago

@nickcolley thanks for the review. tried to address your comments. please let me know if you feel it's good to go or you have any concerns.

NickColley commented 6 years ago

@alex-ju looks great 👍

joelanman commented 6 years ago

Please don't merge while it has my test GA code on it :)

alex-ju commented 6 years ago

@joelanman I'm aware. We can update the tracking code tomorrow. It's up to you when's the right time to merge it, since you opened this PR, just trying to make we have it prepared.

alex-ju commented 6 years ago

@joelanman updated the tracking code. Looking at the Real-time events seems to be tracking well. I'll check again in a few minutes in the Event section to make sure they're stored correctly.

Updated: records well! good to be merged from my side

alex-ju commented 6 years ago

@joelanman thanks for the review! updated.

joelanman commented 6 years ago

@alex-ju it looks like checkFiles still happens after the require I highlighted?

edwardhorsford commented 6 years ago

It might be worth storing the date the user accepted the opt-in. Might be useful if you ever want to decide whether to send a new opt-in request.

joelanman commented 6 years ago

@edwardhorsford I think thats a good idea, but I think we can add it later. For now the file created date should be ok. It raises the question of whether the 'permission proof' living with the user is sufficient. If they ever lost it there would be no proof. I wonder whether in this case we could prove that data is only sent if a user explicitly opts in.

alex-ju commented 6 years ago

@nickcolley, can you give us a hand with the final review? I'm aware you already approved, but there were some small changes since then.