erikrose / parsimonious

The fastest pure-Python PEG parser I can muster
MIT License
1.79k stars 126 forks source link

Use mypyc when building wheels #210

Closed lucaswiman closed 2 years ago

lucaswiman commented 2 years ago

See Gyph's article and the documentation. The basic idea is to annotate some basic types which can allow automatically compiling to more optimized C code that is equivalent to the original python. Basically Cython without a custom DSL, and that works on native types.

In particular, since we know that match_core and expressions.py are hotspots, a few annotations may significantly increase the speed. expr_cache is essentially a Dict[int, Dict[int, Optional[Node]]], so the key lookups may be significantly faster when compiled to C. match_core and a few other methods could be factored out into their own modules to reduce the the surface area that mypyc would touch.

Some downsides are that it would make the build process more complicated, and possibly make error messages more opaque for end users.

Tasks

lucaswiman commented 2 years ago

I did some benchmarking on this and wasn't able to get any noticeable speedup. Maybe if all of expressions.py were written to mypyc's satisfaction, but that seems to be tricky. This may be worth revisiting later once mypyc is more mature and maybe after more of parsimonious is annotated.