search

numbas / Numbas

A completely browser-based e-assessment/e-learning system, with an emphasis on mathematics
http://www.numbas.org.uk
Apache License 2.0
197 stars 118 forks source link

Feature Request: Warn students before submitting of pattern match result #1099

Closed Spruce8913 closed 1 month ago

Spruce8913 commented 1 month ago

We have students answering an equation, representing a real-life constraint in word-problems to practice formalizing stories in mathematics. For example: " you have a budget of {budget} euros that you will spend entirely. You will spend your budget on candy and videogames. If you spend x euros on candy and y euros on videogames, what equation represents the budget constraint?

Answer: x+y={budget} "

We frequently have students answering only the 'interesting' part of the equation, so just x+y, which is marked incorrect. Is there a way to warn students in advance, if they have not used an '=' symbol? This is similar to the functionality of warning about unexpected variable names (see issue here). I have tried to achieve this by adding m_anywhere("=") in the Restrictions tab for what pattern the student's answer must match, but this is only checked after submitting.

More generally speaking, would it be possible to do pattern matching as the student types and warn in advance? If this is too tricky, is there some other way to dynamically warn students about having to enter the full equation?

christianp commented 1 month ago

It would be nice to have more control over warnings. The marking algorithm system at the moment doesn't really let you do anything before the student submits their answer, so the warnings you get at the moment are all built in.

Pattern matching can be quite slow, so I'd like to do some testing before making it happen on every keypress.

christianp commented 1 month ago

@Spruce8913 is this the kind of thing you imagined? The warning is "Your answer is not in the right form:" followed by the pattern restriction warning message given by the question author.

Screencast from 09-05-24 14:10:54.webm

Spruce8913 commented 1 month ago

This is exactly what I'm looking for!

christianp commented 1 month ago

Does this set of options, and label for the setting, make sense?

When to warn the student if their answer does not match the pattern

If On submission is selected, the student will only be told that their answer does not match the pattern after they submit their answer.

If On input, and prevent submission is selected, the student will be shown a warning next to the input box as soon as they enter an expression which does not match the pattern. If they try to submit the answer, it will be marked as invalid.

Spruce8913 commented 1 month ago

If you ask me, I would also allow the option "warn on input, apply partial credit (all other on-input warnings prevent submission)" that I saw in your content. Teachers may want to warn students about a mistake, but still allow them to make the mistake, to discuss it afterwards in person.

I personally would also rephrase the documentation a bit. "On submission" and "On input, and prevent submission" is for not-too-technical readers not superclear. I would use "After submitting", "While typing their answer", ...

christianp commented 1 month ago

@Spruce8913 thanks for that suggestion. The options are now "After submitting", "While entering their answer", and "Prevent submission".

There's documentation at https://docs.numbas.org.uk/en/latest/question/parts/mathematical-expression.html#term-When-to-warn-the-student-if-their-answer-does-not-match-the-pattern