search

doctrine / lexer

Base library for a lexer that can be used in Top-Down, Recursive Descent Parsers.
https://www.doctrine-project.org/projects/lexer.html
MIT License
11.05k stars 60 forks source link

Allow using enums as a token type #77

Closed greg0ire closed 1 year ago

greg0ire commented 1 year ago

Enums allow to avoid having to come up with values to assign for each name (unless you use a backed enum), and will provide more type safety.

There is a small performance impact:

PHPBench (1.2.7) running benchmarks... #standwithukraine
with configuration file: /home/greg/dev/doctrine-lexer/phpbench.json
with PHP version 8.1.12, xdebug ❌, opcache ❌
comparing [actual vs. original]

\Doctrine\Lexer\Benchmark\LexerBench

    benchLexer..............................I0 - [Mo10.680μs vs. Mo10.611μs] +0.65% (±0.00%)

Subjects: 1, Assertions: 0, Failures: 0, Errors: 0
+------------+------------+-----+--------+-----+---------------+-----------------+--------------+
| benchmark  | subject    | set | revs   | its | mem_peak      | mode            | rstdev       |
+------------+------------+-----+--------+-----+---------------+-----------------+--------------+
| LexerBench | benchLexer |     | 100000 | 1   | 1.723mb 0.00% | 10.680μs +0.65% | ±0.00% 0.00% |
+------------+------------+-----+--------+-----+---------------+-----------------+--------------+
greg0ire commented 1 year ago

Pausing this as I discovered a BC-break that affects ORM and possibly more… I think I have to make the switch from array to Token opt-in somehow.