rkmcclung
commented
7 years ago Privacy questions: Does the Privacy questions:
Does the dashboard comply with local privacy laws and institutional policies? Is there a minimum sample size which protects individuals from being identified? Is accessed controlled to the dashboard with appropriate authentication? Are users limited to view only their own data and aggregated cohort data?
Transparency questions: Can stakeholders view data, metrics, models and algorithms? Can they find out who has access to the data and how it's being used? Can stakeholders correct inaccurate data about themselves?
Usability questions: Is the dashboard accessible? Has user testing been carried out?
Actionable questions: Are recommendations provided to the user? Are the recommendations practical and tied to data e.g. online student shouldn't be directed to physical library? Is feedback personalised where appropriate?
Adverse impact questions: Have students been involved in the design? Are appropriate institutional review processes in place? Is the language used sensitive, beneficial, encouraging? Have you looked at what students might be doing to game the system? What are the opportunities for associated human intervention? Has consideration been given to demotivation and the impacts of continual monitoring? Has the data been appropriately contextualised (and not reducing the individual to a metric)?
Stewardship questions: Is attention given to data security, maintenance, retention and access policies? Are data management agreements in place?
Validity questions: Has the data been cleaned effectively? Are the chosen metrics based on evidence? Are continual review processes to place? Is there a review process to ensure quality control of the analytics? comply with local privacy laws and institutional policies? Is there a minimum sample size which protects individuals from being identified? Is accessed controlled to the dashboard with appropriate authentication? Are users limited to view only their own data and aggregated cohort data?
Transparency questions: Can stakeholders view data, metrics, models and algorithms? Can they find out who has access to the data and how it's being used? Can stakeholders correct inaccurate data about themselves?
Usability questions: Is the dashboard accessible? Has user testing been carried out?
Actionable questions: Are recommendations provided to the user? Are the recommendations practical and tied to data e.g. online student shouldn't be directed to physical library? Is feedback personalised where appropriate?
Adverse impact questions: Have students been involved in the design? Are appropriate institutional review processes in place? Is the language used sensitive, beneficial, encouraging? Have you looked at what students might be doing to game the system? What are the opportunities for associated human intervention? Has consideration been given to demotivation and the impacts of continual monitoring? Has the data been appropriately contextualised (and not reducing the individual to a metric)?
Stewardship questions: Is attention given to data security, maintenance, retention and access policies? Are data management agreements in place?
Validity questions: Has the data been cleaned effectively? Are the chosen metrics based on evidence? Are continual review processes to place? Is there a review process to ensure quality control of the analytics?
provpup
Given a scope of thinking about a student counterpart to staff dashboards of predictive analytics
Which ways of communicating the results of learning analytics to students – without intermediation through a professional – are desirable, effective, and free from unacceptable side-effects?
See here for an expansion of the idea https://github.com/LAK-Hackathon/LAK17Hackathon/blob/master/slides/LAK%20Hackathon%20Theme%20and%20Challanges.pdf