search

tree-sitter-grammars / tree-sitter-markdown

Markdown grammar for tree-sitter
MIT License
430 stars 56 forks source link

feat!: update to tree-sitter 0.23 #160

Closed hendrikvanantwerpen closed 2 months ago

hendrikvanantwerpen commented 2 months ago

Tree-sitter released version 0.23 recently. This contains a breaking, but ultimately very good change to how parser bindings work (https://github.com/tree-sitter/tree-sitter/pull/3069). The TLDR of that change is that parsers do not depend on tree-sitter anymore, but instead on a shard (and supposedly very stable) tree-sitter-language crate. As a result, clients are free to chose their tree-sitter version as the parser ABI is supported. Library and parser versions are less tighly coupled, and it should no longer be necessary to move all the parsers to the next tree-sitter version in lock-step to be able to upgrade the library.

Most of the changes in this PR are from running tree-sitter generate. I'll add comments on other changes I did to explain why I thought they are necessary, or where I'm not sure I did it correctly.

Please review carefully, since this project has a non-standard setup, and I may have missed something.

Complications with MarkdownParser

The Rust bindings, besides the gramamr, also provide a MarkdownParser implementation. It needs tree-sitter for its implementation. That means we cannot just remove the tree-sitter dependency like for most grammars.

Leaving the tree-sitter dependency as it was would negate the benefit of the switch to tree-sitter-language and still effectively pin the tree-sitter library for any user of this crate.

My proposal, as implemented in the PR, is to put the convenience parser behind a cargo feature 'parser'. Users that do not want it will get the benefits of the tree-sitter-language approach, while others will have to live (reasonably!) with the restriction on tree-sitter version they can use.

The consequence is that users of the MarkdownParser will need to change their Cargo.toml to enable the parser feature on their tree-sitter-md dependency. Since they will probably need to change their Cargo.toml anyway when upgrading I don't think this is such a heavy burden, but it should be communicated clearly in release notes or something to avoid confusion.

Let me know if you think this is an acceptable solution!

ObserverOfTime commented 2 months ago

Thank you!

hendrikvanantwerpen commented 2 months ago

@ObserverOfTime Thanks for merging these changes! Is it possible to do a versioned release of these as well?