Lex2regex

[Larsouille25]

Improve the lexer using regex so it's more easy and concise.

The performances are too slow :

• A simple input 1 + 1 is lexed in 198ns and with the new regex lexer, it's lex in 1.06ms. It's an augmentation in the runtime of 535253.5% !

• And a much complex one func foo(a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z) a + b + c + d + e + f + g + h + i + j + k + l + m + n + o + p + q + r + s + t + u + v + w + x + y + z takes 18µs with the current lexer and with the new regex lexer, it's lex in 1.12ms. It's an augmentation in the runtime of 6122% !

[Larsouille25]

Unless anyone finds a better way to do this with comparable performance as the lexer in the main branch, this Pull Request is closed

[Larsouille25]

The image shows the benchmark of the current lexer, before it was the lexer with regex and after the lexer that is currently in main. So "criterion" reports -99.980% and -98.567% reduction in execution time.