Semantic parser for code

[luozhiya]

I am writing this Issue to share my thoughts and plans to implement tree-sitter in Mogan.

Currently, Mogan use simple string-matching as programming language highlight that missing semantic tokenization.

I'd appreciate any help/suggestions on this.

Why tree-sitter

• Support 140+ programming language currently.
• Parse is fast, acceptable performance.
• Language specifications are complex.
• Incremental parsing and error recovery.

Useful links

https://tree-sitter.github.io/tree-sitter/using-parsers https://github.com/nvim-treesitter/nvim-treesitter/blob/master/README.md#supported-languages

[da-liii]

tree-sitter support will be added in Mogan Code or Mogan Mark.

As for Mogan Editor, we'd like to make it as slim as possible.

[da-liii]

I would not reject good features implemented in it.

Good features could also be implemented as plugins.

Bringing new dependencies is a significant change, we should be very cautious about those changes.

[mgubi]

A first start would be to understand and document in the Developer manual how parsing is currently done in TeXmacs. And what are the possible approaches compatible with the structure of the editor.