My kit analytics

[swwalker]

As an instructor, I want to review how members interact with my kit so I can improve it.

How might we (the Kits project team) provide useful insights to instructors on how members are interacting with their kit?

Questions What are some possible metrics?

• Some discussion on #77
• Comments below and on #211
• What custom URLs students are creating
• Number of apps per visibility setting
• Activity streams from the apps themselves (@vincentliu98 indicated this would be of interest to students as well)

Assumptions

• That users care about this feature

Hypotheses

[JustinJohnsen]

Partial list from issue #211 that might be relevant here.

Potential metrics to consider for when an app/tool is 'used':

• Lifetime usage: total amount of time users have spent in the app/tool (either histogram or mean + standard deviation)
  • Average usage per user: mean value of lifetime usage among users
• User growth rate: tracking number of users over time
• Installations: number of times Sakai/kit owners install an tool/app in their course/kit
• Active users: number of users entering the app/tool, measured by preferred period (day/week/month)
  • Stickiness: equals daily active users divided by monthly active users
• Session length: total duration of each user session between opening app/tool and closing app/tool (either histogram or mean + standard deviation)
• Time in app per period: total length of time a user spends in an app/tool over a preferred period (day/week/month)

[joliet]

Clarification for this issue: This analytic data would be for the instructor for a specific kit. With our current UI layout, I would expect this feature likely to live under the options/kabob menu in the kit. The analytics page would/could display the data mentioned above, mostly likely about how students are using apps added to a kit. I suspect instructors would want even more data about app-specific usage here, which might require accessing user event data within apps to display within the kit analytic data.

[joliet]

Related to issue: Kits Analytics: build foundation for user and app event or activity tracking- https://github.com/DukeLearningInnovation/kits/issues/441

[joliet]

Documentation for retrieving usage data in apps: Warpwire - https://www.warpwire.com/support/developers/reporting-api/ WordPress - https://developer.wordpress.org/reference/ Box - https://developer.box.com/reference#events Sakai - https://sakai.duke.edu/direct/describe (see sitestats-report) Panopto - https://www.panopto.com/features/integration/integration-apis/ (I will follow up on access) Piazza - They are just now building their public-facing APIs and will share those with us when they are ready. I will up date here when I have more information.

[joliet]

Information on dashboards curated by my colleague Kim Manturuk - https://duke.box.com/s/5uug16q3d2x3zwgzd52moxsfid8n1iau

[joliet]

Feature planning: Important questions we should answer before beginning to build this feature (see link below for more detail).

• Have you identified the main outcomes (changes you desire) and indicators (measures of these changes) for your project, program, team or organisation?
• Is it possible and meaningful to measure the indicators you have chosen on a regular basis? (anywhere from daily to quarterly).
• Define your primary purpose for the dashboard
• Identify the indicators that you will display on your dashboard
• Decide what you would like your dashboard to look like

https://www.betterevaluation.org/en/evaluation-options/data_dashboard