The spec tests covering the error cases are currently exercised as:
a sentinel indicator whether there should be an error
which tokens were parsed before the error
A next level of improvement could be if the reason of error is spelled out in terms of explicit error messages. Some benefits are:
captures the test author's intent.
forgoes the ambiguities; depending on the algorithm used by the implementations:
there could be multiple candidate validations which might be violating and streamlining it would bring about consistency across implementations.
there could be some potentially wrong assumption which may lead to some other ambiguities in implementation; which could be avoided early on.
Current spec test runners will continue to work with binary logic, by checking the existence of error tag (as happening today). Some implementations may chose to align error messages exactly as the spec test describes it for hardening.
For the existing (73) error cases, error messages could be proofread and included in spec test, after collected from certain implementation like libyaml; including the location of error but without the stacktrace e.g.:
Failed to parse YAML. mapping values are not allowed in this context.
in /path/to/file.yaml, line: 2, column: 8
The spec tests covering the error cases are currently exercised as:
A next level of improvement could be if the reason of error is spelled out in terms of explicit error messages. Some benefits are:
Current spec test runners will continue to work with binary logic, by checking the existence of error tag (as happening today). Some implementations may chose to align error messages exactly as the spec test describes it for hardening.
For the existing (73) error cases, error messages could be proofread and included in spec test, after collected from certain implementation like libyaml; including the location of error but without the stacktrace e.g.: