search

Kattis / problemtools

Tools to manage problem packages using the Kattis problem package format.
MIT License
105 stars 72 forks source link

Do not grade empty groups #215

Closed ghamerly closed 1 year ago

ghamerly commented 1 year ago

Fixes #189

This causes verifyproblem to give a verdict of AC with score of 0 to any empty group, rather than calling the grader on an empty group. It also changes the way the default_grader reports a judge error (reporting a score as well).

These changes are mostly only useful for use in verifyproblem with the -d flag, which can filter out test files so that groups appear empty.

niemela commented 1 year ago

Could this ever happen without using -d? I.e. is there something explicitly disallowing empty groups? If not, it might be "more correct" to have the verdict be AC with accept_score instead?

ghamerly commented 1 year ago

@niemela There was previously nothing in verifyproblem that reported a warning or an error if a test case group was empty (actually empty, not just filtered out with -d). With the commit I just added (4e6179e), I have made that situation an error. If you think it's better treated as a warning, I'm open to that.

As far as using accept_score instead of 0 as the default score (when a test group is empty, due to filtering), we can do that easily (I have written but not committed the code that would do that). However, I'm not sure that the semantics are quite correct to apply it this way, because accept_score is used on each test case, not on each test group. Thoughts?