dodona-edu / judge-r

R judge for the Dodona learning environment
MIT License
0 stars 7 forks source link

Command testing by parsing #66

Open gubogaer opened 3 years ago

gubogaer commented 3 years ago

This branch introduces the contextWithParsing function. This context can be used to test intermediate (printed/returned) results that are not saved in an object. This can be done by splitting the student code using comments following a certain pattern. The context function will run the code section by section alternated by the test defined for this section.

gubogaer commented 3 years ago

Please style your code consistently with the rest of the project (space after for and if, space between ) and {, space around ==/!=).

I feel like this doesn't handle mismatches between the exercise's requested sections and the student's provided sections. AFAICT, there is no message when the studnt has superfluous sections, and same for missing? In those cases, a warning to the student also feels appropriate.

Also, IIUC this throws away any code that is not in a section, right? (So all code before the first section, I guess). A student might set up some results/variables there, which could make the rest of the code invalid, even though their solution has correct results when run locally. You could add a warning for that case, but I'd prefer if we executed all submitted code, but just didn't do any tests for a preamble.

bmesuere commented 3 years ago

@gubogaer It seems like some exercises are already using this? We're getting some internal errors. However, @chvp is ill and I'm abroad so we're unable to fix this today.

gubogaer commented 3 years ago

That's correct luckily I made alternative testing methods which i will be using for the upcoming practica today. But thanks for noticing!