Closed thorehusfeldt closed 5 months ago
@mpsijm : thanks for 28f9c457ef27aabe446db00670f0205adc59806e, I was writing almost the same code.
The directory mixed
remains a problem and currently breaks pytest
on hello
.
@thorehusfeldt No problem! :smile: And it looks like the mixed
directory is processed just fine, there was just an empty @EXPECTED_RESULTS@:
string in one of the submissions. :joy: Let's see if that fixes it :slightly_smiling_face: (EDIT: it does! :partying_face: )
I've re-implemented lazy judging and changed it from "stop as soon as parent verdict is known" to "stop when one of the runs gets aborted" (unless -v
or --table
is given, of course). While I was at it, I also changed the displayed run time to be the slowest rejected test case (or, if the submission is AC, the slowest AC test case). Does anybody wants to do a double-check before we merge? :slightly_smiling_face:
Properly determine the verdicts for each testgroup.
verdict.py
verdict.py
verdict.py
, provide method forRun.run
to register verdicts per testcase as those are computed, inferring testgroup verdicts as these become determined. This registry overlaps with the current internal functionality ofVerdictTable
and needs to be unified. This is probably justclass VerdictForTestItem(Dict)
, mappingstr
(testcase and testgroup names) toverdicts.Verdict
.Run.run
to leave the inference of the final (root) verdict toverdict.py
tree
-like overview of verdicts per testcase in UISubmission._get_expected_verdicts
and related code toverdict.py