I'd like to prevent students from being able to 'debug via autograder' by simply submitting and re-submitting their code until they get 100%. I understand that one can limit the visibility of a subset of test cases so that their output and scoring only become visible after the project deadline/grade publication/etc. I'm wondering if, beyond this, it is possible to restrict visibility on test cases based on the submission count. For instance, it would be useful to have test cases either "only visible after the deadline" as well as "visible before the deadline but only on the first (X) submit(s) of the calendar day".
It does not seem, based on the documentation, that this feature is built in to the gradescope-utils functionality for reading the results.json file. If this assessment is correct, I am curious if a hand-crafted workaround is feasible to implement on my end?
I'd like to prevent students from being able to 'debug via autograder' by simply submitting and re-submitting their code until they get 100%. I understand that one can limit the visibility of a subset of test cases so that their output and scoring only become visible after the project deadline/grade publication/etc. I'm wondering if, beyond this, it is possible to restrict visibility on test cases based on the submission count. For instance, it would be useful to have test cases either "only visible after the deadline" as well as "visible before the deadline but only on the first (X) submit(s) of the calendar day".
It does not seem, based on the documentation, that this feature is built in to the gradescope-utils functionality for reading the results.json file. If this assessment is correct, I am curious if a hand-crafted workaround is feasible to implement on my end?
Thanks in advance for your help!