Regrade still applies penalty for "Adaptive mode (no penalties)" behavior

[Willem3141]

When running a regrade on a quiz that has the "Adaptive mode (no penalties)" behavior, then for CodeRunner questions, the regrade ignores the "no penalties" setting, and instead applies the penalty regimes in the individual questions.

(Background: we ran into this problem in practice when regrading a quiz using GraphChecker questions suddenly miscalculated all grades. After experimentation, we found out that it affects CodeRunner too.)

Reproduction steps:

• Make a quiz with the "Adaptive mode (no penalties)" behavior.
• Add a simple "hello world" CodeRunner question; keep the default "10, 20, ..." penalty regime.
• Go to student role and attempt the quiz (note: preview does not work, needs to be an actual attempt). Answer incorrectly, click "Check", answer correctly, click "Check", submit the attempt, and then go back to teacher role.
• Go to Results > Grades, set "Attempts from" to "all users who have attempted the quiz", and verify we got 10.00 points.
• Click "Regrade all" and verify regrading changed the grade from 10.00 to 9.00.

The core of the issue seems to be that both "Adaptive mode" and "Adaptive mode (no penalties)" in CodeRunner are handled by the same class, namely qbehaviour_adaptive_adapted_for_coderunner. This class then decides, based on the $preferredbehaviour flag, whether to apply penalties or not. However, as far as I understand the Moodle code, the question engine constructs a new behavior based on the class name stored with the attempt, instead of based on the behavior stored in the quiz settings (see here). Therefore, penalties are always applied.

I don't know a way to fix this instead of just splitting the qbehaviour_adaptive_adapted_for_coderunner into two separate classes, one with and one without penalties. But hopefully a more elegant solution would be possible?

[trampgeek]

Many thanks for pointing this problem out, Willem. I wasn't aware of it because we virtually never use Adaptive mode (no penalties).

I'm very reluctant to clone and tweak qbehaviour_adaptive_adapted_for_coderunner to a version for adaptive-no-penalties. But I'm not sure what else to do. Perhaps Tim Hunt can suggest something?

[trampgeek]

Hi again Willem. Sorry for the lack of progress here but I simply don't have an easy solution. Since I never use the adaptive no penalties mode, it hasn't been a problem for me personally, though we are thinking of switching to that mode this year. Am I right in thinking that if you also set the penalty regime to 0, there's no problem? It's a hacky workaround but I'm afraid it just might have to suffice. It would be nice to bring up a warning message if a teacher selects that mode, but my code doesn't get called when a teacher changes a quiz mode, and it wouldn't be appropriate to embed a warning in the output that the students see.

[Willem3141]

Hi Richard, you're right that setting the penalty regime to 0 makes the problem go away. This can also be done afterwards: set the penalty to 0, then regrade, and the grades will be correct. (This was also the workaround we applied to fix the grading of the incorrectly-regraded questions. Our Moodle admin kindly assisted by running a database query to set the penalties to 0 for all affected questions.)

We have updated our version of the plugin to set the penalty to 0 by default for new questions, to try to minimize the risk that teachers will run into this problem. I think we cannot do much more here, unfortunately.