search

rwthanalytics / moodle-local_learning_analytics

Learning Analytics for Moodle
GNU General Public License v3.0
20 stars 11 forks source link

readme

Learning Analytics for Moodle

This project offers a privacy-friendly Learning Analytics solution for Moodle. The plugins integrate into Moodle courses and offer statistics to teachers and students.

Download the plugin from the Moodle plugins directory

This Learning Analytics solution consists of two plugins.

To keep documentation in one place, you find all documentation in this repository.

Installation

Requirements:

Download

First, download the dependency (moodle-logstore_lanalytics) and place the files in the admin/tool/log/store/lanalytics folder.

$ cd MOODLE_PATH
$ cd admin/tool/log/store
$ git clone https://github.com/rwthanalytics/moodle-logstore_lanalytics.git lanalytics

After that repeat the steps for this plugin and place the files in the local/learning_analytics folder.

$ cd MOODLE_PATH
$ cd local
$ git clone https://github.com/rwthanalytics/moodle-local_learning_analytics.git learning_analytics

The end result should look like this:

Now visit the Moodle administration page or the Moodle starting page and Moodle should detect the new plugins and install them. Alternatively, you can install the plugins via CLI. As both plugins are very modular, the local plugin comes with several subplugins (of type lareport and lalog)

Activating the logstore

After installation you need to enable the logstore plugin.

  1. Go to Moodle Site administration page
  2. In the Plugins tab, scroll down to Logging
  3. Click on Manage log stores
  4. There should be a row for the installed logstore plugin with the name Learning Analytics Log
  5. Click on the eye icon (👁) to enable the log store.

The log store is now activated and will log events.

After installation and activation of the logstore, there should be a link in each course menu called "Coure Statistics" leading to the following page (COURSE_ID is the course id of the corresponding course):

https://MOODLE_INSTALLATION/local/learning_analytics/index.php/reports/coursedashboard?course=COURSE_ID

Import data from logstore_standard

After installation, all statistics are empty as no data has been logged so far. But your Moodle site might log data through Moodle's own logging system, the logstore_standard_log. The logstore plugin offers a simple way to import that data by using the import.php script. It can be called from the shell like this:

$ cd MOODLE_PATH/admin/tool/log/store/lanalytics
$ php cli/import.php

This will immediately start the import process. Instead you can also call php cli/import.php --help to see a list of possible options. You can for example only import the last X weeks or import only events starting from a specific ID to limit the amount of data you import.

Configuration

You should configure both plugins. The logstore plugin has options related to logging data and writing data to the database. The local plugin has options related to displaying the data.

Plugin logstore_lanalytics configuration

You can find the administration page of the logstore plugin at the same position where you activiated it:

Administration -> Plugins -> Logging -> Learning Analytics Log

Options

The logstore has the following options:

Logging everything vs. only a few courses

When deciding on what to log, keep in mind that there are two plugins:

In case a teacher wants to use Learning Analytics while the semester is already running, you better already have activated the logging in before. That's why we recommend to activiate the logging for all courses by setting log_scope to all and then use the options of the local plugin to decide who has access to the user interface.

Role Tracking

There are two settings to define which roles are to be tracked and which not. Specify the role by using the "shortname" (can be found via Site Administration -> Users tab -> Permissions category -> Define roles).

The blacklist has priority over the whitelist. Keep in mind that a user can be a teacher in one course and a student in another course meanining that a user might be tracked in one course while being excluded from tracking in another.

If tracking_roles is not set, all roles are assumed to be tracked (unless roles are given in nontracking_roles). If tracking_roles and nontracking_roles are unset, all roles are tracked.

For an example, let's assume the following settings:

Setting Value
tracking_roles guest,student
nontracking_roles teacher,manager

In words, this setting translates to:

Only track users in the course that have the role guest or student, unless they also have the role teacher or manager.

Plugin local_learning_analytics configuration

The settings page can be found in:

Site Administration -> Plugins tab -> Local plugins category -> Learning Analytics

Options

The plugin has the following options:

Option course_customfield

This feature requires Moodle 3.7. In older version of Moodle, this option will be unavailable. As described above, the option course_customfield adds a Moodle customfield to the settings page of your course. Technical details how this works can be found in the Moodle Wiki on Custom fields API.

Before switching to this option, you should be aware of the following:

Unfortunately, Moodle does not allow to use multi-language strings here, therefore the option will use the primary language of the Moodle platform here. In case you have the multilang filter activated for content and titles, the function will concatenate existing language strings to provide a similar experience to the use of normal language strings. You can always edit the customfield on your own. You find the options under Site administration -> Courses -> Course custom fields. You can change the description of the field as well as the name of the category. But you should not delete the category or customfield and should not add additional custom fields to the category.

Option dataprivacy_threshold

Our plugin logs no personal data. For privacy reasons, you probably still want to restrict when aggregated data is shown. You can do this by setting the dataprivacy_threshold option. This option will hide datasets with less than dataprivacy_threshold data points. Depending on what is shown, the dataset will be completly hidden or shown as < 10 (with the threshold being 10). Whether data is hidden or not depends on the following:

You should ask your data privacy officer what value should be used here. From our experience, 5 or 10 are common values.

Examples

Let's consider an example for each case. Assuming, the value is set to 10, this will have the following effects:

Aggregated Data

In addition, aggregated data (like the number of total clicks on all quizzes) is rounded (down) to a multiple of the threshold. By doing that, the teacher cannot use the aggregated data points to esimated the data points of a specific activity.

Data storage

What is being stored?

When an event is triggered inside of Moodle, the following data is logged by the logstore plugin:

Table: logstore_lanalytics_log

Field name Type Explanation
id BIGINT
eventid INT Type of action, e.g. "Resource viewed"
timecreated BIGINT Date and time, exact to the second
courseid BIGINT Corresponding course
contextid BIGINT Corresponding context, e.g. ID of the resource that was viewed
device SMALLINT Operating system and browser, e.g. "Windows 10" and "Firefox", detailed browser or operating system versions are not stored

In addition, there are helper tables, that do not store data-privacy related data and only exist to speed up queries or to minimize storage requirements.

Table: logstore_lanalytics_evtname

This table serves as a reference for the type of actions. It maps the eventid from the above table to the actual eventname. The table contains only two columns and is simply used to minimize the needed storage space inside the database.

Table: lalog_browser_os

This tables serves as a memory for devices per course. It uses the device from logstore_lanalytics_log and stores an aggregation per course. This is simply done to minimize requests to the logstore_lanalytics_log table.

Resulting storage size

Our Learning Analytics plugins were developed to log as little data as possible and to be as space-efficient as possible. A single table row needs 38 byte. Additionally, you should expect that you need roughly twice as much space after incorporating storage space for database indexes. All other (helper) tables are not worth mentioning and will need less than a few MB of space.

Some real numbers: The plugin has been used for multiple semesters at the RWTH Aachen University. The RWTH has roughly 45,000 students. In one year (two consecutive semesters), about 115 million rows were inserted into the log table. The needed storage was around 6.9 GB (including data and indices).

Data Privacy

This plugin was developed with data privacy in mind. It does not log any user ids. All data is logged anonymously. As this plugin logs no personal data, you don't need the consent of users to log the data.

After consultation of our data privacy officer, the following information was added to our data privacy statement (DatenschutzerklÀrung):

Im Rahmen von Learning Analytics werden anonymisierte Statistiken zum Zugriff auf LernrÀume gespeichert. Bei jedem Aufruf in Moodle werden dabei folgende Daten ohne einen Bezug zu Nutzenden geloggt:

  • Typ der Aktion (z.B. ob ein Quiz durchgefĂŒhrt wurde oder ein PDF runtergeladen wurde)
  • Uhrzeit (sekundengenau)
  • ID des Lernraums, in dem die Aktion durchgefĂŒhrt wurde
  • Betroffener Kontext (z.B. die ID des Quiz, das gestartet wurde)
  • Genutztes Betriebssystem (z.B. Windows oder Linux) und genutzter Browser (z.B. Firefox oder Edge), detaillierte Versionen werden nicht gespeichert

SĂ€mtliche Daten werden anonym gespeichert und lassen keinen RĂŒckschluss auf Nutzende zu. Die Statistiken können in den teilnehmenden LernrĂ€umen ĂŒber einen Link in der Navigation von allen Teilnehmenden der Veranstaltung abgerufen werden. Aggregierte Daten werden daher in den Statistiken erst dann angezeigt, wenn mindestens 10 DatensĂ€tze vorhanden sind. Andernfalls wird nur "< 10" angegeben.

Access / Capabilities

There is a single capability local/learning_analytics:view_statistics that decides who is allowed to view the course statistics. By default, the following roles have the cabability (as defined in access.php):

That means that, by default students and teachers both have access to the statistics. This was done on purpose for maximal transparency.

Development

Language support

The plugin comes with German and English language files.

Changelog

Every change is documented via GIT. In addition, we create a summary for every version in separate changelog files. All important changes will be documented there. We follow the guides from keepachangelog.

Each of the plugins has a separate changelog and separate versioning. You can find the changelogs of the projects here:

Third-party libraries and resources

The following third-party libraries and resources are used in this project:

See CREDITS for more information including full licenses.

Contributing

Checkout the contributing guide.

Citing in Papers

The Learning Analytics for Moodle plugins were developed by Thomas Dondorf at RWTH Aachen University in the context of his PhD dissertation:

Learning Analytics for Moodle: Facilitating the Adoption of Data Privacy Friendly Learning Analytics in Higher Education

If you use this project as part of your research or want to reference it, please use the following BibTex:

@phdthesis{Dondorf:844544,
    author       = {Dondorf, Thomas}, 
    othercontributors = {Nacken, Heribert and Persike, Malte},
    title        = {{L}earning Analytics for {Moodle}: Facilitating the Adoption of Data Privacy Friendly Learning Analytics in Higher Education},
    school       = {Rheinisch-WestfÀlische Technische Hochschule Aachen},
    type         = {Dissertation},
    address      = {Aachen},
    publisher    = {RWTH Aachen University},
    reportid     = {RWTH-2022-04002},
    year         = {2022},
    doi          = {10.18154/RWTH-2022-04002},
    url          = {https://publications.rwth-aachen.de/record/844544},
}

Sample:

Dondorf, T. (2022). Learning Analytics for Moodle: Facilitating the Adoption of Data Privacy Friendly Learning Analytics in Higher Education (Dissertation). Rheinisch-WestfÀlische Technische Hochschule Aachen, Aachen. https://doi.org/10.18154/RWTH-2022-04002

Provided by & License

The plugins are provided by:

Lehr- und Forschungsgebiet Ingenieurhydrologie - RWTH Aachen University Center fĂŒr Lern- und Lehrservices - RWTH Aachen University

The software is licensed under the GPL.