Open jamestrew opened 8 months ago
I'd like to offer help if there's any interest in that. I have some experience with parsers, mostly parser generators though. I also maintain https://github.com/GregRos/parjs which is a parser combinator library for javascript / typescript.
Some projects publish two grammars, such as https://github.com/tree-sitter-grammars/tree-sitter-markdown
Maybe this could be a good option for this project as well. I'm thinking a new tree-sitter grammar could be created for https://luals.github.io/wiki/annotations/
Any thoughts?
I actually started this a few months back but it's been on hold for a while. Trouble is, the new LuaCATS syntax can be pretty whitespace sensitive and it's a hassle integrating it into the existing parser. And I'm unsure if just writing an annotation-only (and not parse lua code) tree-sitter parser or even not using tree-sitter entirely would be better.
I've also been following neovim core's rework of their docgen and integration of LuaCATS (and talked a little with Lewis on the core team who's been working on this). When I initially created this issue, there was still some ideas of using tree-sitter but since then, Lewis has written a LuaCATS parser using lua and LPEG but told me there aren't plans to make this accessible to the wider neovim ecosystem. At least at the time of the convo several months before more recent changes. Not sure if these plans have changed since.
But at this point, considering the work Lewis has done, it might be easier to fork the work done by Lewis. Or at least revisit this discussion with him.
It seems like what we want should be done in two steps:
I guess the hard part is 1 (parsing), so for that, I'd like to offer two alternative options to writing our own parser.
Use the grammar from https://github.com/tree-sitter-grammars/tree-sitter-luadoc. I believe the grammar is accessible from https://www.npmjs.com/package/tree-sitter-luadoc as an npm package. I think this is what my neovim uses to highlight luacats annotations already, and in my experience it has worked well for that.
Use the lua_ls project's "export documentation" feature to get access to documentation as JSON https://luals.github.io/wiki/export-docs/
Here's some thoughts about each option:
Idea 1
Idea 2
Idea 1 sounds better, easier to migrate. I wanna keep most of the docgen code here, it has some small issues (most of them are related to the current parser) so it should be straight forward to migrate. The only question is if we want to do this here or in a separate repository, because this was always an alternative lua parser and now switching the parser to different one is also kinda weird.
I'd like some input from @tjdevries
I don't know that I have a strong opinion on it right now, I would have to read quite a bit more of the links in the thread
I considered tree-sitter-luadoc as well but I have a few reservations about using it directly
@brief
, @tag
means even more parsing on topBut at this point, considering the work Lewis has done, it might be easier to fork the work done by Lewis. Or at least revisit this discussion with him.
Spoke with Lewis about this again and he's in support of someone productizing a lot of the work his done for neovim core's docgen into something more fit general use. I think I'll give this approach a try unless there's some objection.
Is a desire to convert the emmylua based type annotation grammar to luals' luacats grammar? Considering the growth in popularity of luals' luacats, I think this would be nice.
If there is, I can take a stab at it. I have some experience writing tree-sitter grammar.