Enchanced Moodle grading support

[jon-betts]

The problem

• The LTI spec says all grades are between 0.0 and 1.0. In the UI we interpret this as 0 - 10
• Moodle has three different ways of grading which are all incompatible with this view:
  • None - No grading at all
  • Point - Between 0 and x, where x ∈ [1 ... 100]
  • Scale - A pick-list of discreet options like "Good" or "Terrible"

See also:

• How does grading work in Moodle with LTI?
• https://github.com/hypothesis/lms/pull/1262 - Proof of concept for Moodle scale (and other) handling

Motivation

• We want our product to be as compelling to Teachers as possible
• Teachers should not have to use their LMS in a special way to make our product work
• In cases where our product directly doesn't work (None or Scale) our product is less appealing
• In cases where we do work, but don't match the values in the LMS (Point) we require the user to do extra work to translate, which is unpleasant and could be confusing
• All of these cases could lead to a negative view of our product as "janky" or unfinished
• Work done for Moodle should point a way forward for other LMS specific integration which could benefit products with a wider install base (Canvas etc.)

Requirements

Must have:

• When a teacher launches an assignment with the different options our interface changes to match
• For Point we show the correct Out of number as the maximum and validate accordingly
• There must be a way to configure any required options to make this work (API keys etc)
• The solution cannot be unacceptably slow or error prone
• We must have a good method for encapsulating Moodle specific behavior so the code base doesn't become very Moodle specific because we happen to be working on this
• There must be graceful fall-back when this functionality fails or is not available in a particular install

Should have:

• For None we show student navigation but no ability to submit grades
• For Scale we show a pick-list with the correct options (e.g. "Pass/Fail", "A/B/C", "Good/Bad/Rubbish")
• When things go wrong we should be able to inform the user so they can fix it
• We should have considered efficiency, so we do not slow down the service or make unnecessary API calls
• We should have considered whether elements of this solution can be extended to other LMS's (Having a record not out of 10 seems like a common requirement)

Could have:

• The ability for teachers to directly fix issues in a slick way (but as long as it's possible at all, this isn't totally necessary)
• If parts of the code base are re-contextualised by this work they should be renamed. For example this is likely to differentiate between student navigation and grading. If these concepts are a single thing right now they should be differentiated.

Won't have:

• Any complex queuing methods for optimising calls out-of process (It's a nice idea but simpler methods will likely do for now)
• Any functionality for other LMS's above and beyond what we do right now (even if we have nicely set ourselves up for it)

---

The solution

The main components of a solution to this are:

• Grading methods - Presentation and transport of Point, None, Scale data
• Feedback - Letting the teacher know what's happening when things fail
• Assignment information - Retrieving, and storing Moodle "course module" information
• Token management - Storing the web service token

Each of these can be tackled with a certain level of fidelity. The proposal is to tackle the work in five phases:

Phase 1 - Better defaults

• Return a hard coded grading configuration object to the UI
  • This will contain a minimum and maximum grade (0-100 for Moodle, 0-10 for others)
  • We will present and capture grades in this format
  • The decision of what to present will be LMS specific
  • Scaling will be handled by the back-end not the front-end

Benefits:

• We can present different defaults for different LMS's
• Tech: Control of grading and scaling is moved to the back-end
• Tech: We begin the process of a plugable grading model in the front-end

Tickets:

• https://github.com/hypothesis/lms/issues/1267 - Phase 1: Allow maximum and minimum grading values to be controlled from the back-end

Phase 2 - Tech demo

• Web-service token will be manually added to the database
• We will make syncrhonous calls to the Moodle WS API for every required call
• We will use the results to configure the Point values being sent to the UI
• Graceful fallback on errors

Phase 3 - Point grading MVP

• Present errors to the user (call to action?)
• Add ability to manage the WS token in app

Benefits:

• A Moodle teacher (with help from an Admin) can now benefit from accurate representation of the points they setup in a course
• We present errors to the user to allow them to request help when things aren't working
• We can use some of these errors as calls to action:
  • "You could have a better Moodle Grading experience. Click here to find out how!"
• Tech: We have proved out the API interaction

Phase 4 - Quick gains

• Extend configuration to None and Scale type grading
• Direct users to the configuration introduced in Phase 3 (where appropriate) to help them to fix errors

Benefits:

• A Moodle teacher can how use None and Scale type grading correctly
• We provide better guidance about how to fix things, rather than just that they need to be fixed

Phase 5 - Efficiency

• Implement caching of results so that calls to the Moodle WS API are no longer 1:1 with calls to our app

Benefits:

• Users will get a snappier experience with these features enabled
• Tech: We won't melt if this becomes popular

---

Follow on work

• Blackboard
  • Also has "Grading disabled"
  • Supports a number between 0 and anything. No default, but can we read this?

[mkdir-washington-edu]

I received feedback from an instructional designer at an EDU that relates to how we handle "Scale" grading. I'm adding it here since Scale is listed in the "Should have" section, but if you'd prefer this be a separate issue, please let me know.

The ID pointed out that his higher ed institution had switched to competency-based grading, and that this type of grading is already common in K-12 institutions across many LMSs. His vision was that we would have already built a scale into the Hypothesis grading bar that would approximate scales used in other institutions. I think this would be difficult (while most competency-based grading models use 4 different scale points, some folks are starting to use 5), but I'm not sure what a feasible technical solution is.

He recommended any Scale implementation actually has teachers see a scale with labels and select from them, as opposed to, say, entering a numerator for x out of 4 or x out of 5 points.

Notes are in an internal note in this ticket, as most of this was relayed on a call: https://hypothesis.zendesk.com/agent/tickets/7885