search

DOMjudge / domjudge

DOMjudge programming contest jury system
https://www.domjudge.org
GNU General Public License v2.0
739 stars 258 forks source link

Output vizualizer #2744

Open vmcj opened 1 month ago

vmcj commented 1 month ago

I tried to implement https://github.com/DOMjudge/domjudge/issues/367. This is still a bit rough but I rather first get some feedback instead of continuing with this if this is not what @eldering envisioned.

The main problem I currently have is which testcases to visualize, do we only want the failing one and if so only the wrong-answer one or do we just want to try all of them and accept that the script might fail as the team output might not conform to the expected output. (For example: the team might have stopped halfway through the problem or might still have debug output etc.)

The visualized team output for boolfind is ofcourse not very helpful but as this is only a demo this is the best for now.

When implementing this I found out we could just always visualize as a task with lower priority and update the visualization task with the judgehost which should have the needed files. I wrongly assume that the file will always be at the judgehost to keep this simple but that should still be fixed if we want to continue with this.

vmcj commented 1 month ago
image
meisterT commented 1 month ago

Getting this error when clicking the "visualize" button: image

meisterT commented 1 month ago

Do we limit upload filesize anywhere?

meisterT commented 1 month ago

One implication of current implementation (which is reasonable) is that if you shut down a judgehost, you are not going to get the visualization. We should probably think about a good way to message this to the user.

meisterT commented 1 month ago

Out of scope for this PR, we could have a toggle at the problem level to visualize the output right after judging a test case (perhaps filtered down by verdict, e.g. only for wrong-answer).

vmcj commented 1 month ago

One implication of current implementation (which is reasonable) is that if you shut down a judgehost, you are not going to get the visualization. We should probably think about a good way to message this to the user.

That problem is also there for the debug package I think. It wouldn't be that hard to extend here to optionally calculate the result on the testcase but if a submission is non-deterministic we would get a different visualization as what the original team-output had. Your remark https://github.com/DOMjudge/domjudge/pull/2744#issuecomment-2429912162 would prevent the problem but would give a problem when visualization takes a some time and there is a backlog.

vmcj commented 1 month ago

Out of scope for this PR, we could have a toggle at the problem level to visualize the output right after judging a test case (perhaps filtered down by verdict, e.g. only for wrong-answer).

I assume still with the priority of first the submission work and only if there is nothing do the visualization or do you want to extend the queued work to have a {Run+Visual}[x] with x as the length of the worklist?