Do not ignore leading & trailing whitespace by default

[niknetniko]

As long as I could go back in git history, leading and trailing white space has been ignored by default. Basically, what happens when evaluating results is expected_stdout.strip() == generated_stdout.strip().

Recently it was said that this should not be the default, as we want to be strict by default.

However, this would make the simple case more difficult. For example,

- tab: "Testen"
  testcases:
  - stdin: "Jan"
    stdout: "Hallo, Jan!"
  - stdin: "Piet"
    stdout: "Hallo, Piet!"
  - stdin: "5236-hallo!"
    stdout: "Hallo, 5236-hallo!!"
  - stdin: "'wereld'"
    stdout: "Hallo, 'wereld'!"
  - stdin: '"wereld"'
    stdout: 'Hallo, "wereld"!'
  - stdin: "    "
    stdout: "Hallo,     !"

would need to become:

- tab: "Testen"
  testcases:
  - stdin: "Jan"
    stdout: "Hallo, Jan!\n"
  - stdin: "Piet"
    stdout: "Hallo, Piet!\n"
  - stdin: "5236-hallo!"
    stdout: "Hallo, 5236-hallo!!\n"
  - stdin: "'wereld'"
    stdout: "Hallo, 'wereld'!\n"
  - stdin: '"wereld"'
    stdout: 'Hallo, "wereld"!\n'
  - stdin: "    "
    stdout: "Hallo,     !\n"

A possible solution would be to add trailing newlines automatically to the test suite for stdout and stderr (and maybe stdin). However, we maybe also need a way to disable this behaviour then.

[pdawyndt]

After giving this some thought, here's what I would like to suggest:

• use a parameter (suggested names sanitize-output or autocomplete-final-newline) that adds \n at the end of stdout and stderr (does not apply to a returned string-value) if it doesn't already end with a newline; the default value of this parameter is True to make stdout and stderr automatically compliant with the C-adage that all text files should end with a newline, without these newlines cluttering the test specification
• because the previous silently adds newlines to stdout and stderr by default, it is quite safe to migrate to exact comparison between expected and generated stdout and stderr without stripping by default (without breaking too many exercises that already use TESTed)

I guess this is the better of worlds where we do exact matching by default, but with silent addition of final newlines in case they are missing from the test specification (or without the need to clutter the specification with newlines after single-line outputs. This might also alleviate the tricky ways in which YAML treats newlines in multiline strings.

[pdawyndt]

Also relates to https://github.com/dodona-edu/dodona/issues/5168: automatic insertion of final newlines if missing (by default) would also resolve this Python Tutor issue (although it would seem more sensible if Dodona also wouldn't ever strip the final character (newline)).

[pdawyndt]

I now see that my suggestion is also what @niknetniko suggested (but didn't read the original post until the very end).

[niknetniko]

The conclusion of talking it over IRL is the following:

• We will disable left stripping by default.
• We will keep right stripping by default.
• We will add an option to the DSL to changing the strictness.

[niknetniko]

I think the argument was made to in any case only strip newlines, not all white space, as this would break some exercises (e.g. those where a grid should be made). This seems reasonable to me.

[pdawyndt]

Here's another reason why I would by default append a newline to stdout and stderr the string included in the test specification does not end with a newline: to follow the C/POSIX convention in the expected output displayed in Dodona:

[niknetniko]

From #486:

That the expected/generated values are not the same is a big argument in favour of doing strict comparison in my opinion, so I would not be against making the validations strict (as the main argument against it previously was that the status quo caused no problems if I recall correctly).

[pdawyndt]

Here's an example that shows how confusing the newline-difference is for students, as it distracts from the true issue: