dodona-edu / universal-judge

Universal judge for educational software testing
https://docs.dodona.be/en/tested
MIT License
9 stars 4 forks source link

Unify return values in DSL #427

Closed niknetniko closed 10 months ago

niknetniko commented 10 months ago

Fixes #403.

This changes the DSL to have one attribute for return values: return. Then, to decide how to handle it:

  1. If the value is an untagged YAML string; interpret it as the string (Python) representation.
  2. If the value is an untagged object; interpret is as the advanced output channel object (for oracles).
  3. Else, interpret it as a value, like the current return behaviour.

Additionally, it introduces an additional tag, !v to explicitly mark a value as a YAML value. Some examples:

In most cases, this will make the test suites easier, as both statements, expressions, return values and oracle arguments all use the Python syntax by default.

This change is not backwards compatible. I'll need to go though the repositories to see where we must change usages:

pdawyndt commented 10 months ago

Maybe we should use a more explicit name then !v to improve readability.

niknetniko commented 10 months ago

Maybe we should use a more explicit name then !v to improve readability.

I went for !v to reduce clutter; e.g. if you have a string return value everywhere, that's a lot of times "value" will be there.

However, better readability is also something we want, so perhaps it is better to be explicit.

niknetniko commented 10 months ago

We now support both !v and !value