Tabular unicode lexing support

Dervall / Piglet

An easier to use parsing and lexing tool that is configurable in fluent code without a pre-build step.

binarysculpting.com

MIT License

91 stars 11 forks source link

Tabular unicode lexing support #32

Closed Dervall closed 12 years ago

Dervall commented 12 years ago

The lexer should work in tabular form with unicode. This will give a slight performance hit, but not too bad.

Refactor the transition tables to work with character ranges instead of plain characters.

Dervall commented 12 years ago

This is fixed for 1.3.0. Also the unit testing now constructs lexers with all possible configuration options to make sure that we don't get lexer issues.