search

maths / moodle-qtype_stack

Stack question type for Moodle
GNU General Public License v3.0
142 stars 149 forks source link

Ready to grade input validation markker #638

Closed aharjula closed 4 months ago

aharjula commented 4 years ago

Lately, students have gotten too fast, they fill in inputs and click the "check" button so fast that the automatic validation does not have the time to happen. In the case of multipart questions this leads to problems as the validation may be able to mark some parts as valid leading to a situation where the students check button clicking results in "partially correct" feedback even if the answers are generally correct. These students tend to react fast to that feedback and instead of trying the check button again they simply start writing complaints to the course personnel.

Naturally, this would not be a problem if all the students were taught the validate+check process of STACK questions but this is something that has been missed during the current chaos and the same chaos also leads to load for the systems which may increase the auto-validation time just enough for this to happen more often.

I have been thinking that the default validation messages for inputs should include some form of a symbol signalling to the students that this input was not yet graded as it was not known to be valid, or that it is not graded because the other inputs required for grading are not yet filled. I think that we actually had something like that in STACK2 days or maybe it was part of the PRTs then.

Does anyone have any ideas about what those symbols should look like or any other ideas about this? From my point of view the input validation box just needs to have a small symbol in some corner (maybe at the place where the loading animation appears) to signal four states:

  1. This has been graded in this form. As part of some PRT...
  2. This has been validated and is ready to for grading, press check to get points.
  3. This is valid and all but is not enough for grading please fill more inputs. Only for the case where none of the PRTs this belongs to have the full set of inputs.
  4. This is invalid! For those that cannot even bother to read the red box with the details.

The compact validation might have similar problems, but I have no idea about how it should display this.

christianp commented 4 years ago

Why is it possible to check an answer that hasn't been validated? Could the checking process also validate, if the automatic validation response hadn't been received when the check button was pressed? Or, could the check button be disabled until the answer is validated?

Relying on the user to do things the right way, even if you explain it very clearly to them, will always result in problems.

timhunt commented 4 years ago

The button should be disabled until valiation has finished. I have been meaning to implement that for years.

sangwinc commented 4 years ago

Good idea Tim! The current behaviour is a slight hangover from an explicit two step "validation>assessment" check. Disabling the button while validation happens is a very good solution.

christianp commented 4 years ago

Rather than disabling, after the button is pressed you could wait until the validation has finished before sending the check request. That allows the student to press the button as soon as they want to, and the 'checking' process just seems to take longer.

aharjula commented 4 years ago

So disabled during the AJAX-validation? That seems sensible.

To Christian, we validate in many ways, there is the technical validation that always happens but the other part is the key here, we want the student to validate that we have interpreted their answer correctly. And we must be certain that they have seen it before we are sure that they have validated it. If we were to continue with the checking without giving them the chance to change their input it would not be a good thing in a system where they may lose points due to penalties related to errors. So pressing the check can only grade after that validation has happened.

aharjula commented 4 years ago

To me, Tims disabling of the check button during AJAX-validation solution seems like the best idea and solves the issue pretty much completely.

What do you think about those symbols I mentioned? Should we bring in a way for the students to see if they need to fill in more before any grading can happen and maybe those states which say that this value (or combination of values) was already graded so nothing new will happen during "check". That latter might mean that we need to read the history from the step-data though.

sangwinc commented 6 months ago

@EJMFarrow in the next version, please can you disable the "check" button while AJAX-validation is going on?

EJMFarrow commented 5 months ago

I've set up a pull request for the check button. There's still Matti's original question about validation indicator icons to be dealt with, though. Any opinions?

EJMFarrow commented 4 months ago

I'm going to close this now. The change to disable the check button while the AJAX request is taking place will be in 4.7.0. This deals with the issue. If anyone is still keen to add extra indicator icons then please get in touch.

sangwinc commented 4 months ago

Thanks @EJMFarrow