Keywords design

[tjvr]

• Consider making keywords explicit, rather than implicit. Currently it's not obvious that the keyword transform is happening!
• Consider adding an ignoreCase option, which is currently missing from Moo (see #46). Note that this can only apply to keyword literals, not RegExps; we can't compile arbitrary regexps to their case-insensitive equivalents! The value of the emitted token would always be the originally specified literal (i.e. case-normalised).
• Allow renaming token types based on their keyword—this would allow writing parsers which switch only on the type of tokens, not their value
• Optimise recognition of keywords (generate a switch, rather than using a dict)

[tjvr]

Specifying keywords should probably look like this, to match the { tokenType: matchPattern } convention moo uses for specifying lexers.

{
    'kw-while': 'while',
    'kw-else': 'else',
}

(With an accompanying public helper function for generating such dicts, from a naming function and a list of keyword literals.)

[tjvr]

@nathan You designed most of the Moo API; do you have any suggestions for this? :-)

[deltaidea]

If token types are gonna be renamed, will this impact error messages? E.g. Expected foo but found bar in nearley.

[nathan]

we can't compile arbitrary regexps to their case-insensitive equivalents!

Well…

Maybe you shouldn't…

Consider making keywords explicit, rather than implicit. Currently it's not obvious that the keyword transform is happening!

Perhaps just make it an error for keywords (i.e. literal string matches) to appear after a non-keyword rule. Then it's obvious that they take precedence.

[tjvr]

Keywords are now explicit.