search

joncatanio / cannoli

Cannoli Programming Language
MIT License
771 stars 20 forks source link

Move lexer/parser into own crate #7

Open jared-mackey opened 6 years ago

jared-mackey commented 6 years ago

It would be handy to have the lexer/parser separate from the compiler if possible. That would allow projects to make an interpreted version without having to rewrite the lexer/parser.

joncatanio commented 6 years ago

It was originally split out into it's own crate, but I also provided a lib.rs that would allow one to import the lexer/parser into their project. This would be done by including the cannoli crate in the cargo manifest and imported via cannoli::parser or cannoli::lexer. Splitting it out would be cool as well, I don't mind.

windelbouwman commented 6 years ago

It would maybe make sense to move the lexer / parser into its own crate. I would certainly like that for using it in my project: https://github.com/windelbouwman/rspython Also see another attempt here: https://github.com/shinglyu/RustPython It would be good to join forces somehow. Maybe a seperate crate can be made for lexing, parsing and conversion into an AST. This would make sense to me at least.

On the other hand, splitting a project into subpackages also has it's negative sides. For example, development is more complicated since you have to update two sourcebases and ensure compatibility. For this reason alone, I would not split it up, but rather make it such that parts of the project can be re-used.

pothos commented 6 years ago

Interesting to find more parsers and lexers now. Two years ago I also ported the lexer to Rust https://github.com/pothos/zpaqlpy/blob/master/src/rtok.rs and used lalrpop for a simplified grammar https://github.com/pothos/zpaqlpy/blob/master/src/grammar.lalrpop - having just one well maintained crate would make sense from my point of view.