thSoft / elysium

LilyPond IDE for Eclipse
http://elysium.thsoft.hu
14 stars 3 forks source link

More robust grammar tests #80

Open thSoft opened 9 years ago

thSoft commented 9 years ago

Currently, the regression tests for the grammar check only whether the inputs can be successfully parsed. Some basic assertions about the resulting ASTs are needed, as well as tests for inputs with parse errors.

nittka commented 9 years ago

81 provides a first simple pattern for possible AST-tests. I would not do those on complete snippet files but rather on selected inline code examples. Rather than validating the resource yourself, ValidationTestHelper may be injected in order to assert that there are no issues or issues of a certain type.

nittka commented 9 years ago

I would like to split up the snippets folder in the test project into snippets and "snippetsCurrentlyKnowToFail". Until the complete language is supported, it would be good to know which integration tests are expected to pass (snippets) - even after changes have been made. The new folder could also be used to add test cases that we would like to work on. Once a file passes a test, it moves to the snippets folder - or a distinct regression test case folder in case you want snippets to be a mirror of the snippet library of the current LilyPond release.

nittka commented 9 years ago

Initially, I thought the tests in Grammar.java were broken. The model is parsed there, but there are no checks for errors (running the test incorrectly, I got a couple of them).

Anyway, if it's fine with you, I will move the test cases to GrammarRegressions.xtend.