search

hypothesis / support-legacy

a place for tracking support-related work and projects
3 stars 0 forks source link

Spike: Moodle doesn't allow the editing of Max Grade on an activity once a grade has been given. #182

Closed mkdir-washington-edu closed 3 years ago

mkdir-washington-edu commented 3 years ago

Describe the bug On activities in Moodle the max grade usually defaults to 100 points. On a regular Moodle activity where a student submits work and an instructor grades it the instructor can edit the activity and change the max grade. When they do so they're asked if they want to scale the existing grade to the new max grade.

On Hypothesis-enabled readings embedded in activities* changes to the Max Grade are not reflected in the grading bar. It's unclear whether we can actually do anything about this, or if it's even unique to the Hypothesis tool. This issue is meant to serve as a Spike to see if there is anything to be done.

*This may be true of all activities using external LTI tools, rather than something unique to Hypothesis. I couldn't find anything about this online, but I installed the ChemVantage LTI tool (they offer their development server for LTI testing) in our Moodle instance and I can replicate the error with their tool as well.

To Reproduce

To quickly see this in action:

  1. Log in to Moodle as a teacher at https://hypothesisuniversity.moodlecloud.com/login/index.php
  2. Open an assignment - you can use this one, called Sidebar test - PDF: https://hypothesisuniversity.moodlecloud.com/mod/lti/view.php?id=329
  3. Note the grade is out of 10. Select a student and submit a grade.
  4. Go to the assignments list
  5. Scroll down to the assignment (Sidebar test - PDF) and click Edit > Edit settings
  6. Scroll down to Grade
  7. Change the "Maximum grade" value to something else (like 20)
  8. Re-launch the assignment, and notice that the grade is still out of 10.

Steps to reproduce the behavior: To create and edit a non-Hypothesis activity

  1. Log in to Moodle as a teacher at https://hypothesisuniversity.moodlecloud.com/login/index.php
  2. For steps 3 - 7, instead of creating your own assignment you can use https://hypothesisuniversity.moodlecloud.com/mod/assign/view.php?id=320
  3. Enter a course; I recommend Support 101
  4. Turn editing on
  5. Add a activity or resource
  6. Select "assignment"
  7. Create an assignment
  8. Log in as a student, find the assignment, and submit a text
  9. As a teacher visit the assignment and click "Grade"
  10. Assign a grade to the student submission
  11. Edit the assignment
  12. Change the Max grade
  13. Note the ability to scale existing grades

To create and edit a Hypothesis activity

  1. Log in to Moodle as a teacher at https://hypothesisuniversity.moodlecloud.com/login/index.php
  2. For steps 3 - 7, instead of creating your own assignment you can use https://hypothesisuniversity.moodlecloud.com/mod/lti/view.php?id=329
  3. Enter a course; I recommend Support 101
  4. Turn editing on
  5. Add a activity or resource
  6. Select "external tool"
  7. Create an assignment (make sure to Select "Hypothesis" in the "preconfigured tool" category
  8. Log in as a student, find the assignment, and annotate
  9. As a teacher visit the assignment and using the grading bar to select the student and submit a grade
  10. Edit the assignment
  11. Change the Max grade
  12. Save the assignment, leave, then edit the assignment again
  13. Note the new Max grade did not actually save; the assignment still contains the old max grade.

To create and edit an activity using a different LTI tool

  1. Log in to Moodle as a teacher at https://hypothesisuniversity.moodlecloud.com/login/index.php
  2. For steps 3 - 7, instead of creating your own assignment you can use https://hypothesisuniversity.moodlecloud.com/mod/lti/view.php?id=393&forceview=1
  3. Enter a course; I recommend Support 101
  4. Turn editing on
  5. Add a activity or resource
  6. Select "external tool"
  7. Create an assignment (make sure to Select "ChemVantage" in the "preconfigured tool" category
  8. Create a quiz
  9. Log in as a student, find the assignment, and take the quiz. At the conclusion click "grade this quiz".
  10. As a teacher edit the assignment
  11. Change the Max grade
  12. Save the assignment, leave, then edit the assignment again
  13. Note the new Max grade did not actually save; the assignment still contains the old max grade.

Expected behavior I would expect the Hypothesis-enabled readings to function like a native Moodle assignment, with a change to "Max grade" prompting the user to scale existing grades to the new max grades.

Note that this may be outside of Hypothesis' control; it may be a general Moodle/LTI issue.

Additional context https://app.hubspot.com/contacts/6291320/ticket/335881878/

klemay commented 3 years ago

A couple of notes here:

mkdir-washington-edu commented 3 years ago

@klemay I've researched this and couldn't find anything online about Moodle having this issue with LTI tools in general, although it's always possible I missed a key term somewhere.

I'm happy to make this a spike, but I'm unsure how. I do think it will ultimately turn out to be a Moodle problem and not a Hypothesis problem, but it would be good to verify this. Then I will updated our documentation.

klemay commented 3 years ago

I think this is a duplicate of https://github.com/hypothesis/lms/issues/1169 - I've asked @mkdir-washington-edu to confirm & close if that's the case

mkdir-washington-edu commented 3 years ago

This is not a duplicate of that issue. That issue is about how Hypothesis displays the max number of points in the Hypothesis grading bar. This issue is about the actual max number of points the assignment is worth within Moodle.

klemay commented 3 years ago

@mkdir-washington-edu Given that this is reproducible for non-Hypothesis tools, before handing this to eng as a Spike, I think we should try to get confirmation from Moodle re: whether there is anything an LTI provider can do about this.

Can we get in touch with an LMS admin at a Moodle school who could help us get this question in front of Moodle support?

mkdir-washington-edu commented 3 years ago

@klemay I've asked the user for some help in contacting Moodle. I agree it makes sense to see if they can help us answer the question of whether this is even something an LTI tool can affect.

mkdir-washington-edu commented 3 years ago

@klemay Here's what I got back from the Moodle admin:

I posted this to the CLAMP Slack workspace. One person responded:

It’s an issue with Moodle LTI last I knew. There are some possible workaround with like categories and such, but it’s not great.

Another responded:

Shouldn’t LTI only ever return grades as a % anyway? What I mean is, I think that LTI can only return grades as % anyway, so the main problem is on your Moodle. If you change the max grade, is there a way to ask the LTI to push grades again? I know there is on a couple LTIs I work with, like a resync grades button

So it sounds like this is expected Moodle behavior. I think we should note "If you change the max grade, is there a way to ask the LTI to push grades again? I know there is on a couple LTIs I work with, like a resync grades button". Obviously this doesn't apply to us now as Hypothesis doesn't store grades so there's nothing to re-sync, but if this is a common solution for other platforms we may want to start storing grades in the future.

klemay commented 3 years ago

Thanks for this, @mkdir-washington-edu! I've put this issue back on the product & bug backlog for consideration.

Given there's nothing we can do unless we start storing grades, I don't see us approaching this one any time soon, just because storing grades opens us up to a whole host of FERPA & other legal issues (and, I believe, would violate some existing contracts).

That's not to say we'll never do it, just that I see this being a "long term" item.

mkdir-washington-edu commented 3 years ago

Seeing as this is expected Moodle behavior I don't think we can build anything to change it. Closing.