search

okpy / ok-client

A Python client for the OK autograding system
https://okpy.org/
Apache License 2.0
57 stars 42 forks source link

Hide a testcase from the student #411

Open eldos-dl opened 4 years ago

eldos-dl commented 4 years ago

I have noticed that students can see all the test cases, that are being checked by looking at the tests folder. Is there a way to hide the tests from the student?

Sumukh commented 4 years ago

If you want students to reason about the expected output before seeing them, you can use the "locking" feature of OK (See the paper here for it's effectiveness: http://denero.org/content/pubs/las15_basu_unlocking.pdf)

If you're ok exposing the input but not the output, you can wrap it an another layer of hashing. Here's one way we've done that: http://denero.org/content/pubs/las16_sridhara_fuzz.pdf

If you fully want to hide the existence of a test case, you'll have to use server side auto-grading (and not client side scoring). It's pretty hard to fully hide what a program is executing (because end users control the code)