search

dodona-edu / universal-judge

Universal judge for educational software testing
https://docs.dodona.be/en/tested
MIT License
9 stars 4 forks source link

Merge contexts across tabs #329

Closed pdawyndt closed 10 months ago

pdawyndt commented 1 year ago

Merged processing of contexts is now restricted to contexts in a single tab. This gives an overhead for exercises with multiple tabs that could be eradicated if contexts could be merged across tabs.

niknetniko commented 10 months ago

Currently, if compilation fails, we try to compile each "execution unit" individually (this used to be per context, but since we have runs, it is per execution unit). However, with this change, an execution unit is potentially a complete test suite, rendering the fallback compilation mode moot.

Thus, should we try again by compiling each tab separately, or should we compile each context separately? The first is far more efficient, but the second one is "more" correct. A third alternative would be to first try compiling per tab, and then per context. If we want to make things complicated, we could even determine this per tab, e.g:

  1. Try compiling everything together, if this fails:
  2. Try compiling each tab. For each tab, if this fails:
  3. Try compiling per context.

We could thus have a scenario where the first tab is compiled as a unit, after which tab 2 is compiled per context.