Implement syntax highlighting (both old/transitional syntax)

[theY4Kman]

Brief

As the core of any good IDE, the syntax-highlighting must be squeaky clean. The core of syntax-highlighting in IDEA relies on a JFlex lexer generated from a .flex definition, and a parser generated from a .bnf grammar.

Details

NOTE: until I feel my way around a bit, this ticket will be a bit short on details

Basically, the lexer should be pretty simple — not relying on backtracking or any other bad-worst-case feature of regular expressions. Good lexing should be quick and predictable. Preferably, some automated performance tests would verify speeds on some large source files.

The grammar should produce parse trees that very closely resemble what the SourcePawn compiler produces. This can be verified with the PsiViewer plugin. I would imagine Jetbrains provides some handy automated testing tools for verifying parse trees.

[theY4Kman]

Jeez, ya know, I just sort of assumed there was an official grammar out there :P I've been working on a first draft, aiming to parse the kitchen sink with the ugliest, messiest, rubber-bands-and-tape grammar that does the job; then starting fresh with a solid foundation (and much better naming).

This is what I've done so far (during late nights). Things like arbitrary blocks don't exist, yet. Complex logical expressions (&& ||) aren't a thing, yet, either. Nor if/else, switch, for, while, do/while (SP has those, right?), and on and on. And the naming is horrendous :P But it can sorta parse!