search

Dervall / Piglet

An easier to use parsing and lexing tool that is configurable in fluent code without a pre-build step.
binarysculpting.com
MIT License
91 stars 11 forks source link

Lexer unicode support #11

Closed Dervall closed 12 years ago

Dervall commented 12 years ago

The lexer should include unicode support. I'm unsure how this might be best accomplished, I guess one solution could be to handle unicode characters as character combinations, i.e. two normal characters in a row.

Dervall commented 12 years ago

This is feasible now that we can run the lexer directly from the DFA and NFA. Tabular form will not be possible using unicode, at least not now.

Dervall commented 12 years ago

Fixed for 1.3.0