lw
commented
11 years ago I don't like this idea that much because:
- It's difficult to decide when to "outdate" results: when a manager is changed, when a testcase is added/removed, when a time/memory limit is changed, etc.
- Since it's the TaskType's duty to decide which managers to use at compilation and execution stages and hence there's no way to know which results are affected (e.g. if the Pascal manager is changes all C/C++ submissions are outdated too).
- We also have no way to know if only evaluation is outdated or if compilation is too.
- Even this way we could get a lot of false positives: a bug in a manager/testcase could affect only a handful of submission and "outdating" them all could be useless (and possibly a waste).
- We've always given great powers (and great responsibilities) to contest administrators, because they know best. This seems to go in the opposite direction.
Anyway, if at the end we decide we want something like that, I'd prefer to do it in a way that was proposed somewhere some time ago: store a "hash" for each result that is derived from the digests of the files that were used (+ other useful information) and use that to determine if a result is outdated.
giomasce
stefano-maggiolo
If an admin changes something in a task (read: dataset after they are merged), there is no way to discern the submission results that were computed with the new task/dataset data and those computed after. Proposal: add an "up_to_date" field in the submission results table to be flipped when something changes in the task/dataset, and allow admins to re-evaluate only those submission.