-
Currently the generated lexers can only open files by name and the files must be mmap-able.
Therefore it would be nice to be capable of lexing from other sources, such as strings, sockets, pipes, std…
-
When lexing a String Value we can have three different error types:
- unterminated string
- unexpected escape character
- unexpected line terminator
If there is a unexpected escape character or …
-
### Description
I recently upgraded from FsLexYacc 10.0 to the latest 11.3.0. After the upgrade, parsing a comment line `// ä` now fails with "unrecognized input". I have made no changes to the lex…
-
- [x] delete peek2 https://github.com/llvm/llvm-project/pull/99790
- [x] delete consumeLabel https://github.com/llvm/llvm-project/pull/99567
- [ ] #100462
- [x] #101035
- [ ] Improve https://gith…
-
Currently, we have a bunch of logic duplicated in the lexer/parser for raw-strings. That's because normal raw-strings are 100% handled in the lexer. However, interpolated raw strings have to go thro…
-
-
In short, instead of passing a token or regex pattern, I want to pass a function that would return Option.
such as:
```rust
#[derive(Logos)]
enum Keyword {
#[token(let)] Let,
#[token(if)…
-
- [ ] block names can not have spaces
- [ ] `````` gives ```Lexing error in '(GetPosAlt()```
- [ ] `````` gives ```Lexing error in '(agl_dist_valid&&(agl_dist_value```
- [ ] New GCS crashes wi…
-
After [P2621R2](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/p2621r2.pdf), which is a defect report, the following program is ill-formed in C++ (UB beforehand):
```
#define X '\N'
int…
-
Currently, our parser directly operates on a stream of characters, i.e. tokenizing is built into the parsing. The idea would be to make our own token type, write a lexer that transforms a character s…