Implement partial scoring

[meisterT]

• With partial scoring testcases are grouped in testcase groups.
• Each testcase group has a number of points assigned.
• If the submission solves all testcases in the group, the points for that group are awarded.

Then there are two variants:

1. The points for a submission is determined by the sum of the awarded points per testcase group. The points of the problem are determined by taking the maximum of the points of all submissions to that problem.
2. Multiple submissions can solve different testcase groups, so you take the maximum per group over all submission and then sum up.

If we would implement the first variant, the second variant could be worked around by splitting it up into multiple problems.

The kattis problem format suggests adding a testdata.yml per testcase group: https://icpc.io/problem-package-format/spec/problem_package_format#test-data-groups

[meisterT]

From https://cms.readthedocs.io/en/v1.4/Configuring%20a%20contest.html#computation-of-the-score:

“Use the sum over each subtask of the best result for that subtask across all submissions” is the score mode that follows the rules of IOI since 2017. It is intended to be used with tasks that have a group score type, like “GroupMin” (note that “group” and “subtask” are synonyms). The score on the task is the sum of the best score for each subtask, over all submissions. The difference with the previous score mode is that here a contestant can achieve the maximum score on the task even when no submission gets the maximum score (for example if each subtask is solved by exactly one submission).

[alfphaderp]

In each case, how will penalty time be calculated?

I think this is probably the most pressing issue for our contest in particular. We're currently simulating the second variant by having multiple problems, but this means that penalty time gets multiplied for each testcase group, which is kind of awkward. I think having a single penalty time for the entire problem calculated as max(all submission times) + X * max(# wrong submissions for each testcase group) would make most sense.

In this example, this team gets 63 + 67 + 70 = 200 penalty for problems 5, 5b1, and 5b2 despite submitting the same code 3 times in succession after seeing the verdict for each previous testcase group. I think an ideal situation would be if they would be able to submit one time, use checkboxes to select all testcase groups, and then get a single 63 as their penalty time.

[eldering]

Related to my comments in https://github.com/DOMjudge/domjudge/issues/2525#issuecomment-2091757210, I'm not a big fan option 2 to let the score for a problem depend in such a complex way depend on the score of each individual submission. I'd rather stay with: the score for a problem is always the maximum over the scores of each team's submissions for that problem.