Data/Logs on app usage

[stianr]

As a product owner, I want to know as much as possible about who uses my site, when, and how they interact with it, and with what kind of device/OS/Browser, to allow me to understand how both lecturers and students interact with my site, how much different parts of the site are used (e.g., revise a previously take quiz), and potentially identify weak links or problems with the flow of the user journey (e.g., by looking at where people exit the app).

Discussion in #586 indicated that we might be able to use google analytics and google tag manager to get some insights. We might also be able to the general logs, either using an existing package to parse them into something useful, or writing something myself.

So here it would be good to get an idea of the options available, and the time they'd take to implement, if anyone has any thoughts...

[sohilpandya]

Hi @stianr, We have had a chat with @Danwhy and we will be spending today trying to get Google analytics setup for you.

There are a couple of levels we need to think about, the first implementation will contain the basic analytics and that is our first point of focus, once we’ve done that, the idea is to add a data layer for the individual actions for detailed insights.

[iteles]

@stianr We will be looking at this on Monday so if would be great if you could provide us with a concrete list of what you'd like us to track so we can fit as much of that in as is practicable.

[stianr]

The more data the better - I'm sure we'll think of things we want to be able to do that I haven't thought of now. Not sure if this is just for the app, or for the app and website - probably useful for both.

But I think there are three types of thing that would be useful to track:

Visitor information

• Location/IP, HTTP referrer, device, OS, browser, screen resolution, status (lecturer/student), time spent on site, pages requested. Use: Checking who is using the app, from where, and how many people, to give us an overview of our (potential) users. Popularity of different browsers OS, to help target future development (e.g., do we need to test on Windows phones or is that a tiny minority of visitors, what screen resolutions do we need to consider). Checking popularity of pages such as review my answers / revise quiz

Timings (app only)

• Timestamps from either when quizzes are added, edited, run or better still when each question is presented
• Timestamps for when students submit an answer, or load a question page Use: Work out when specific quizzes were run, to help work out patterns of usage, and times when demand is likely to be high. Work out participation rates for quizzes, how long lecturers allow for each question, and how quickly students respond, and how many miss the cutoff, for future development and to share with users. Identify users who login but do not complete a quiz.

Navigation

• Path taken from arrival on site to exit. Not sure how that would be summarised. Use: Look at areas where people drop out to identify issues with app. Look at how people use the site, whether they log in when they are not taking the quiz, and why. Examine what people look at on the website (e.g., information about learning or pricing, or practicalities).

[sohilpandya]

@stianr, we have now added Google Analytics to the app. This can be accessed via https://analytics.google.com, it is currently linked to a local version of the app but once we've merged everything to the live app, then it'll reflect the live version.

We've set it up so that every page view is logged and visible in the analytics.

On top of this, we've also added custom tags for the events that you've mentioned above both for students and lecturer. These can be accessed via Behavior -> Events -> Overview

We are currently logging these events when:

• lecturer adds a quiz
• lecturer edits a quiz
• lecturer runs a live quiz
• student submits an answer
• student reviews/revise a quiz

Google Analytics can be a handful as there is so much going on but all of the information such as devices used, user location, screen resolution is all logged and can be viewed.

To log in to the google analytics dashboard, you'll need to use the gmail account quiz.cityuni{at}gmail.com

[iteles]

Moved to Backlog as there is further work to be done beyond sprint 8 timeframes.