zesterer / chumsky

Write expressive, high-performance parsers with ease.
https://crates.io/crates/chumsky
MIT License
3.53k stars 147 forks source link

Pratt parser only implemented for tuples A..Z #658

Open mlgiraud opened 1 month ago

mlgiraud commented 1 month ago

I really appreciate the pratt parser feature. However, it is currently limited in the number of operators you can implement with it, since impl_pratt_for_tuple!(A_ B_ C_ D_ E_ F_ G_ H_ I_ J_ K_ L_ M_ N_ O_ P_ Q_ R_ S_ T_ U_ V_ W_ X_ Y_ Z_); only supports up to 26 operations. Maybe this can be fixed by using a proc macro to implement the Pratt parser for tuples of size N, where N can be supplied in some way by the library user? I'm no expert on macros though. A quick fix that would suffice for most cases would probably be to just double the tuple length?

zesterer commented 1 month ago

Hmm, you're right that this is a bit of an annoyance. choice supports > 26 because it allows for nesting, but of course the same does not apply for pratt. Is this an issue for you currently?

zesterer commented 1 month ago

An alternative solution might be to allow pratt to work with arrays/vectors as choice does. This would require boxing the operators, but that's not necessarily a problem.

mlgiraud commented 1 month ago

Yeah it is an issue, but i just created a fork for now and just added some more idents from AA to ZZ. I didn't look at the implementation yet, but would it not make it harder for the compiler to optimize, if we allowed using a vec and boxed parsers?

zesterer commented 1 month ago

It would make it a bit harder for it to optimise most likely, yes. The compiler might still be able to perform devirtualisation if it's smart enough though.

mlgiraud commented 1 month ago

i guess making it a vec might make things simpler. Maybe we can try this and check the performance impact? One could also add a small macro along the lines of vec! to create a vec of boxed parsers, so that they don't have to be boxed manually? If i find the time, i can try to propose some changes if you'd like.

zesterer commented 3 weeks ago

I made an attempt to implement this a while back but ran into issues. I might attempt it again later.