SPT syntax tests succeed even if the parse result is ambiguous

molenzwiebel commented 3 years ago

Summary

A parse test that yields an ambiguous AST succeeds, whereas the same test in Spoofax 2 would fail.

What you did

SDF3:

Term.Lit = INT
Term.Add = Term "+" Term

SPT:

test ambiguous [[
3 + 1 + 2
]] parse succeeds

What you expected to happen

The test fails.

What actually happened

The test succeeds.

Context

Spoofax version: 0.11.6
Operating system & version: All

Additional information

Especially annoying because the WebLab tests for CS4200 still use Spoofax 2, so students end up having a solution that works in their own Eclipse but fails on WebLab (rightfully so, since it is ambiguous).

Gohla commented 3 years ago

I'm changing it to:

ParseSucceeds: parsing succeeds (produces AST & no error messages), is unambiguous, does not recover.
ParseFails: parsing fails (doesn't produce AST or produces error messages)
ParseAmbiguous: parsing succeeds (produces AST & no error messages), is ambiguous.

Do you think that is ok?