Open evouga opened 1 week ago
This is a really good cleanup. Some fairly minor comments above.
One use case that we discussed, which is not currently well-supported, are problems where we want to display truncated or too-small sample inputs in the problem statement which (intentionally) do not pass validation.
Do we still care about this case?
If so, then in sample/statement
maybe we want to allow:
.in
(required, input-validated, available for download by default), the actual input for the test case.displayed_in
(optional, not input-validated), the input shown in the problem statement.ans
(required), the answer_file
passed to output validation.displayed_ans
(optional, available for download by default), what is shown in the problem statement and passed to the output validator as team_output
(currently called .out
)EDIT: although, this still doesn't really allow for too-small inputs...
@mzuenni, wdyt? Does this address https://github.com/Kattis/problem-package-format/issues/338? In particular, is/was your only concern the testing tool itself, and are you happy/OK with the data/sample/{.,statement,download} structure?
looks good, the structure is fine :)
I've done a pass on this section. The requirements here are quite complex so please double-check everything carefully.
In particular:
sample/statement
andsample/download
..interaction
and.in
and.out
is provided, only the former is shown (is this true)?sample/statement
. I assume that was unintentional?.interaction
files by default (is this true?).ans
files.validator_args
indata/sample
also apply todata/sample/statement
(is this true?)