search

haskell / alex

A lexical analyser generator for Haskell
https://hackage.haskell.org/package/alex
BSD 3-Clause "New" or "Revised" License
298 stars 82 forks source link

Parse numbers in Alex's parser, not tokenizer #201

Closed andreasabel closed 2 years ago

andreasabel commented 2 years ago

In different contexts within Alex's surface syntax, something like "2340898" might be a string of characters or a number. The contexts are are only distinguished at the grammar level, not the token level, so this more or less (we could very layer-violation-y tricks) precludes lexing entire number literals.

Instead of a number token, we have a digit token. This we treat as "sub-token", making a DIGIT | CHAR non-terminal we use everywhere we want to parse a character.

For number literals, we just parse a non-empty string of numbers, and the left recursion makes the * 10 elegant.

Fixes #197

This is an updated version of #200. Had to open a new PR since I cannot push to the repo. This PR should be open to edits.

andreasabel commented 2 years ago

Why do I get no rule to make target 'issue_197.n.run'? https://github.com/andreasabel/alex/runs/4914025395?check_suite_focus=true#step:22:110

And weird why CI does not run here.

andreasabel commented 2 years ago

@Ericson2314 Maybe you can push the change to #200 and see if CI runs successfully there... The change is $1 * 10 + $2.

Ericson2314 commented 2 years ago

Yeah very weird, I will try.

andreasabel commented 2 years ago

Closing in favor of #202.