effekt-lang / effekt

A research language with effect handlers and lightweight effect polymorphism
https://effekt-lang.org
MIT License
292 stars 14 forks source link

Parser performance: rewrite parser using different combinator library #412

Open dvdvgt opened 2 months ago

dvdvgt commented 2 months ago

By investigating the time spent per phase using PR #411, it became apparent that the parser is a major bottleneck of the compiler. The parser regularly occupies more than 50% of the total time spent compiling (for more complex/realistic programs with imports). This offers a massive opportunity for speeding up the compiler.

In my opinion, we have three options here. Use a parser combinator, parser generator library or a completely handwritten parser.

Using a handwritten parser can be a lot of work, and given the experimental stage Effekt is in, also in terms of syntax, it may be hard to make changes to a handwritten parser down the road. After a quick search, I did not find any actively maintained, reasonably popular parser generators for Scala. This leaves us with parser combinators, and there are a few candidates:

  1. https://github.com/j-mie6/Parsley
  2. https://github.com/com-lihaoyi/fastparse

fastparse promises some nice speed-ups.

b-studios commented 2 months ago

Minor comment: we do not use scala-parser-combinators but the Kiama library. The thing that I am most afraid of is that we introduce a lot of regressions when it comes to recording source positions on AST nodes.

dvdvgt commented 2 months ago

Thanks, I changed it.

Maybe we should first add some more tests for the parser, so that we can more easily detect regressions.

b-studios commented 2 months ago

The problem is the positions which mostly die up in error messages and with LSP. Adding those tests could be a lot of work.