dodona-edu / universal-judge

Universal judge for educational software testing
https://docs.dodona.be/en/tested
MIT License
9 stars 5 forks source link

Re-add config inheritance #538

Closed niknetniko closed 2 months ago

niknetniko commented 2 months ago

441 enabled re-using parts of a test suite by using YAML anchors, aliases and merge keys. It also removed our own inheritance support for config options, as using this inheritance means there is no way of specifying "ignore". For example, if a tab has a config option for stdout, it will be inherited by all testcases inside the tab, unless you explicitly override it.

While at the time, the idea was that this would be confusing sometimes, it does cover the most common case of "apply this config to everything". Therefore, this PR re-adds the support.

The means specifically that the following test suite will work as expected:

- tab: "echo"
  config:
    stdout:
      normalizeTrailingNewlines: False
  testcases:
    - expression: 'echo("Hello world!")'
      stdout: 'Hello world!'
    - expression: 'echo(42)'
      stdout: "42"
    - expression: 'echo("A\nstring\nwith\nnewlines")'
      stdout: "A\nstring\nwith\nnewlines"

Without this PR, you would need to apply the config to all testcases individually.