trampgeek / moodle-qtype_coderunner

CodeRunner: A moodle quiz question type that runs student-submitted program code in a sandbox to check if it satisfies a given set of tests.
GNU General Public License v2.0
207 stars 120 forks source link

Allowing "same response" in the question settings #194

Open XiRouz opened 8 months ago

XiRouz commented 8 months ago

Hello, dear CR developers.

We are using CodeRunner a lot, it is an extremely useful tool, so we push the limits of it all the time. We've developed a new usage for CR, long story short, we use CR to check if a student has done some actions on remote server, or to reset the state of our practical work on remote server. It is done via ssh and python, so basically the code is stored in question and never changes, students rarely type any answer that matters for our checks, with some exceptions.

I believe that all of the problems listed below could be easily avoided if we had an option to accept and run/precheck same responses for questions, and that is a simple workaround, that may be useful to other people too.

Problem 1: when a student re-initializes his practical work via a question in quiz, he needs to type different responses every time, 'cause Moodle won't run any code with precheck if response is the same.

My workaround for problem 1 is my UI plugin for CR that hides textarea for answer and inputs current epoch timestamp in it, so the response is different every time and the precheck is running necessary code. However, every time a student submits a precheck, the textarea automatically changes and after the student got his result of precheck, when a student leaves re-init question page, qbehaviour runs process_save, and saves the new response with TS because of the difference between previously submitted answer and refreshed non-submitted one, while also making a new step and setting the state to TODO. This results in a need to run the same code on finishing the quiz, and consumes extra time.

This could also be solved through some refactoring of my JS UI plugin, but I am not sure if I can reliably understand the state of question through JS UI plugin code, or if I can listen for precheck submission and add a new timestamp to answer's textarea right before the question is submitted. If that is possible, please tell me how can I do it.

Problem 2: Sometimes our CR server doesn't respond in time, resulting in timeout, and when a student needs to type an answer, and the answer isn't changing (like a path to a file, or file rights code (777 etc)), the precheck is not running, because response is the same. Students, who are unfamilliar with the nature of qbehaviour for coderunner, sometimes run into this problem, and the solution is simple, yet unobvious.

If you have any suggestion on how those problems can be resolved in another and already available way, please share the solution.

Thank you!