Dashboards - going beyond single activities

[houshuang]

Questions:

• How can we represent data about students spanning multiple activities?
  • One question could be what Yannis has pointed out, the difference between a "logical/pedagogical" activity and an activity as encoded in FROG... if we have a chat, and a collaborative editor open at the same time, it's logically "one activity", but it will generate two dashboards, and there is no way of doing inference (people who chat more, contribute more in the collaborative editing, etc).
  • But more generally, we are often interested in how students' state change through the graph, we might be interested in completion rates as percentage of "still active" students, which depends on data from previous activities etc. We have talked about representing some simple statistics like "completion" directly onto the graph view, but we would probably want to go much further - what are good ways of generalizing this? Do we have a new kind of dashboard plugins that get access to all the logs from the entire graph? And do we push towards standardizing some metrics from various activities (any kind of graded activity could generate a success score from 0 to 1, some kind of "interaction"/"contribution" measure, etc)?

[aperritano]

I like the idea of standardization of metrics. Are there different classes of metrics? e.g., logistical -- completion rates, pedagogical -- the kinds of responses students input, etc...

[yandim]

Sorry for my late response... The issue pointed out by Stian is very important to me.

It is very common that instructors have in mind a set of connected activities, something like a "logical activity" or "subgraph", an intermediate level between an activity and a graph. An example may involve what occurs in the first level of a pyramid, in which student read resources, create a concept map and provide an ordered list of prioritized list. Then, they move to the second level of the pyramid, in which they try to find a consensus on the concept map and the prioritized list of items.

In the current conceptualization, a dashboard provides indicators (I prefer to think more of a container of data and indicators, and not a visualization) based on actions within a single activity (while in the example we have three connected activities). Thus, there should be a way to connect the three "dashboards-data containers" taking into account their location - connections in the graph. Thus, one might provide a first step of connecting the learning design product (the graph) and the learning analytics "container", and then try to create indicators that span the connected set of activities (the "subgraph"). This might be done through an action at the FROG editor, where the designer defines that these three activities are connected and that learning analytics should be obtained for the whole set.

Regarding the metrics, i.e the learning analytics indicators, a combined one might be very interesting.