search

haskell / alex

A lexical analyser generator for Haskell
https://hackage.haskell.org/package/alex
BSD 3-Clause "New" or "Revised" License
298 stars 82 forks source link

Parse numbers in Alex's parser, not tokenizer #200

Closed Ericson2314 closed 2 years ago

Ericson2314 commented 2 years ago

In different contexts within Alex's surface syntax, something like "2340898" might be a string of characters or a number. The contexts are are only distinguished at the grammar level, not the token level, so this more or less (we could very layer-violation-y tricks) precludes lexing entire number literals.

Instead of a number token, we have a digit token. This we treat as "sub-token", making a DIGIT | CHAR non-terminal we use everywhere we want to parse a character.

For number literals, we just parse a non-empty string of numbers, and the left recursion makes the * 10 elegant.

Fixes #197

Ericson2314 commented 2 years ago

If you could review this, @andreasabel, that would be great!