Closed Larsouille25 closed 1 year ago
Unless anyone finds a better way to do this with comparable performance as the lexer in the main
branch, this Pull Request is closed
The image shows the benchmark of the current lexer, before it was the lexer with regex and after the lexer that is currently in
main
. So "criterion" reports -99.980% and -98.567% reduction in execution time.
Improve the lexer using regex so it's more easy and concise.
The performances are too slow :
A simple input
1 + 1
is lexed in 198ns and with the new regex lexer, it's lex in 1.06ms. It's an augmentation in the runtime of 535253.5% !And a much complex one
func foo(a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z) a + b + c + d + e + f + g + h + i + j + k + l + m + n + o + p + q + r + s + t + u + v + w + x + y + z
takes 18µs with the current lexer and with the new regex lexer, it's lex in 1.12ms. It's an augmentation in the runtime of 6122% !