-
> I added your 8-bit ANSI into the docs, and it looks to be working well
It doesn't s…
-
Removed proper string lexing with the new nom lexer. Need to reimplement it
-
Although it's often not specified by the grammar, parsers should be careful to consider how whitespace is consumed and different tokens separated lexically. This can get really messy though, and in th…
-
It would be really lovely to have a slightly more resilient _lexer_.
There's two different kinds of problems that the lexer can encounter:
1. "completely recoverable": instead of crashing when enc…
-
I have been mulling this for a while, but the difficulties in fixing #197 made it feel more urgent.
As a (rare) user of Adga, I have been very fond of it's lexing, which seems very simple, and more…
-
Trying to build merlin on ecaml-multicore I get a syntax error because in a pattern
``` ocaml
| [x]-> ...
```
the lexer sees ]-> as a single token instead of two.
-
The default lexer of Chevrotain takes the first match according to the order in which the terminals are defined. This is different to the default lexer of ANTLR, which takes the longest match (IIRC).
…
-
The BCL has had Unicode capabilities quite a while ago in the form of [Runes](https://learn.microsoft.com/en-us/dotnet/api/system.text.rune?view=net-8.0). We should consider using runes for lexing to …
-
The JSX lexing kicks in after the previous token is considered a *expression prefix* (this works for JSX lexing). So for the colon `:`, JSX lexing mode kicks in
```
let x = { a: T) | number;
// …
-
we should allow more than just strings as input.
this will allow stratifying our parsing. so we could make one parser that lexes a string, and then a second parser that parses the tokens from the lex…