I've added the ability to set custom evaluators for problems in CloudCoder. That is, if you don't want to check the results literally or with a regex, you can write your own code to check the results. It only works with python functions now, where the evaluator code must also be written in python. It's backwards compatible, it has the same default evaluator code as the old python checker, so it should give the same results.
I might make it work for C programs/functions too in the future, because I'm teaching C for the rest of the semester.
I've also added some new UI stuff to support this, so all the evaluator stuff in the Edit Problem page is greyed out when the problem type doesn't support custom evaluators.
Hi!
I've added the ability to set custom evaluators for problems in CloudCoder. That is, if you don't want to check the results literally or with a regex, you can write your own code to check the results. It only works with python functions now, where the evaluator code must also be written in python. It's backwards compatible, it has the same default evaluator code as the old python checker, so it should give the same results.
I might make it work for C programs/functions too in the future, because I'm teaching C for the rest of the semester.
I've also added some new UI stuff to support this, so all the evaluator stuff in the Edit Problem page is greyed out when the problem type doesn't support custom evaluators.