search

Kattis / problemtools

Tools to manage problem packages using the Kattis problem package format.
MIT License
105 stars 72 forks source link

verifyproblem: if the language of an output validator is not from an allowed list, warn the user #211

Closed ghamerly closed 1 year ago

ghamerly commented 1 year ago

Some judging environments (such as those used in Kattis) do not have all languages available. For example, a sandbox for judging a Python 3 submission might not have Java available. If a problem uses an output validator written in Java, and the Java runtime is not available, the output validator would not work. So we want to warn the user that only a subset of languages are recommended for output validation. These recommended languages include Python 3 (because verification runs in Python), and compiled languages like C and C++ (because they do not need additional libraries or runtimes).

simonlindholm commented 1 year ago

Kattis runs the output validator in the submission's sandbox/Docker image? D:

A more exact error message would be useful, telling you why it's not recommended and which language to use instead, as well as a documentation update so you can avoid the situation ahead of time.