search

eclipse-langium / langium

Next-gen language engineering / DSL framework
https://langium.org/
MIT License
665 stars 61 forks source link

Implement async parsers #1352

Closed msujew closed 4 months ago

msujew commented 5 months ago

A follow up on https://github.com/eclipse-langium/langium/pull/1306.

Implements the basics to create an async parser based on node worker threads. While in theory this should be faster than the sync parser implementation (due to parallelisation), it might be slower on startup due to the time it takes for all the workers to start up. Once the threads are all up and running, it is indeed faster. The break even point is fairly late though, roughly after parsing ~2 million LoC.

The main benefit of this change is that it allows parser cancellation by simply killing workers. The cancellation happens after a timeout, since it needs to take the time it takes to create a new worker into account. The default is 200ms, but can be arbitrarily changed by adopters.

Most of the change is actually just the new Hydrator service which tries to dehydrate/hydrate AST nodes to be processed by the structured cloning algorithm of workers.

sailingKieler commented 5 months ago

One more thing: I prefer to block merging this PR until #1258 is merged, and to fix the conflicts as part of this PR. So maybe I was a bit to quick with my approval 😅