search

apluslms / mooc-grader

Automatic assessment framework compatible with A-plus LMS.
14 stars 30 forks source link

Questionnaire freetext question: ignorerepl option must support Scala 3 output #134

Closed markkuriekkinen closed 2 years ago

markkuriekkinen commented 2 years ago

From the O1 course:

The "ignorerepl" switch for questionnaire items means that students are allowed to answer entire lines of text copy-pasted from the Scala REPL even though the question just asks for a single element from that line.

For example, a question may ask the students to use the REPL to determine a certain integer result. If the question is marked as "int-ignorerepl", then the student may either answer with the integer itself (say, 123456) or they may paste in the entire line from the Scala REPL that they used to determine the result, such as this:

res0: Int = 123456

Now, the above was thrown in years ago when the questionnaire directive was first created for O1’s purposes. It’s a bit of a hack, since it’s language-specific, but it has worked fine. However, since the recent introduction of Scala 3, the REPL output has changed, and there’s a "val" in front of such outputs. Like this:

val res0: Int = 123456

So I would be happy if that format was additionally/alternatively supported (i.e., ignored for grading purposes).

(I suppose the best solution would be to make this configurable somehow, so that it wouldn’t be Scala-specific. But I’d be OK with a quick-and-dirty fix too.)

Private ticket: https://rt.cs.aalto.fi/Ticket/Display.html?id=21634

I think this regex is the one that grades these inputs. Currently, it doesn't seem to allow whitespace (val res0 contains a space) in the start before the colon :. https://github.com/apluslms/mooc-grader/blob/5b8b120453552f9b8a1a31e742ee14d240605054/access/types/forms.py#L478-L482

markkuriekkinen commented 2 years ago

I think this regex is the one that grades these inputs.

More precisely, it removes the REPL part from the value so that only the actual value is compared to the model solution. For example, both res0: Int = 123456 and val res0: Int = 123456 should become just 123456 which is compared to the model solution. The model solution defined by the teacher in the configuration could be simply 123456.