zom-lang / zom

Zom is a Ahead Of Time compiled system programming language.
Other
4 stars 2 forks source link

Remake of the Lexer #54

Closed Larsouille25 closed 11 months ago

Larsouille25 commented 11 months ago

This branch entirely remakes the lexer, now it's more clearer, clean and spans on Token are where they should go.

Larsouille25 commented 11 months ago

Before merge make some benchmarks and try optimizing it if possible

Larsouille25 commented 11 months ago

Benchmarks

keep in mind :

feat/remake-lexer -> 05332ea1dce5fc53e0ea8b9368b48ed1d9c9ffab before optimizations

lexer 1 + 1             time:   [175.45 ns 175.74 ns 175.96 ns]                        
                        change: [-55.168% -55.030% -54.893%] (p = 0.00 < 0.05)
                        Performance has improved.

lexer func(26 args) addition of the 26 args                                                                             
                        time:   [26.362 µs 26.380 µs 26.401 µs]
                        change: [+103.12% +106.23% +108.99%] (p = 0.00 < 0.05)
                        Performance has regressed.

main -> d9f05a8afa8ca29cca34068822e0e74528238ff5

lexer 1 + 1             time:   [327.50 ns 329.10 ns 331.03 ns]                        
                        change: [-3.1258% -1.5861% -0.0316%] (p = 0.05 < 0.05)
                        Change within noise threshold.

lexer func(26 args) addition of the 26 args                                                                             
                        time:   [6.1743 µs 6.1951 µs 6.2225 µs]
                        change: [-3.3750% -2.6583% -1.8773%] (p = 0.00 < 0.05)
                        Performance has improved.

I will for now not doing any kind of optimization(s) to focus on the implementation of a working compiler. But it needs to be optimized.