RagnarGrootKoerkamp / BAPCtools

Tools for developing ICPC-style programming contest problems.
GNU General Public License v3.0
50 stars 22 forks source link

generators: all `testdata.yaml` should be listed #283

Closed thorehusfeldt closed 1 year ago

thorehusfeldt commented 1 year ago

Currently, the generators script is lenient about existing testdata.yaml files in data. It notes their existence with a yellow warning

skipped: testdata.yaml

but otherwise leaves them alone.

Since testdata.yaml critically modifies the behaviour of input validation (and potentially grading), I think we should be more strict and require these files to be explicitly listed in the generators.yaml and otherwise removed. At least the tool should be as prissy as with unlisted testcases and in angry red complain something similar to

Testcase not listed in generator.yaml (delete using --clean).

Trying to think of a situation where I’d want unlisted testdata.yaml floating around in the data directory: Maybe somebody writes a tool that auto-generates testdata.yaml. Then at least I’d expect

testdata.yaml:

in generators.yaml to signal explicitly that the file exists. That’s still too implicit for my taste, so I’d prefer something like

testdata.yaml: swedish_olympiad_settings/testgroup_settings.yaml
RagnarGrootKoerkamp commented 1 year ago

See d6731f00163e0274f6b926cc79ffb9d303362ab7 (numbering branch).

Made it so that unlisted testdata.yaml always error and are removed on --clean. An empty testdata.yaml: key currently just implies the absense of such a file.

I'd prefer to not yet add any specific handling for now -- this can be done when it's needed.