Use tree-sitter for parsing

[kskyten]

There is an emacs package for interfacing with the tree-sitter parser. It supports Julia.

[gcv]

I know about Tree-sitter, it's really cool. Unfortunately, it's also yet another heavy dependency, since it requires a native module. It's bad enough that Snail requires vterm, and I want to eliminate that dependency in the future. To justify adding Tree-sitter, it has to offer substantially more than the current parser.

[kskyten]

Yes, the heavy dependency is unfortunate. I would like to be able to generate docstrings from function signatures and getters from a struct definition. Can that be achieved with the current parser? Better documentation lookup would be really great, but this is probably easier to handle from the Julia side. How does the communication work currently? Would it make sense to use https://github.com/kiwanami/emacs-epc?

[gcv]

I would like to be able to generate docstrings from function signatures and getters from a struct definition. Can that be achieved with the current parser?

Not easily. The parser generator library I used, Parsec, is easy to use but slow. I had to drastically simplify the rules of Julia's syntax to achieve adequate performance. This included swallowing parentheses whole, so extracting function parameter lists will require workarounds.

Better documentation lookup would be really great

What do you have in mind?

How does the communication work currently? Would it make sense to use https://github.com/kiwanami/emacs-epc?

I was not aware of emacs-epc. The current Snail protocol does something similar, but is probably more primitive. It sends expressions in the target language over a TCP connection and calls eval on them, i.e., Julia sends Elisp to Emacs, and Emacs sends Julia code to the Julia process. The setup works well enough for Snail, however. If it turns out to add substantial value, I am open to switching to emacs-epc, but it will involve a significant rewrite.

[kskyten]

For the documentation lookup I was thinking of something along the lines of https://github.com/tkf/InteractiveCodeSearch.jl. Would be nice to be able to parse/infer types of arguments to narrow the search as well.

[gcv]

It's not as nifty as InteractiveCodeSearch.jl, but the Emacs built-in xref-find-apropos does something similar, and Snail interoperates with it. Use helm-xref or ivy-xref or something similar for a nicer interface. You should update Snail before trying; I just made some small changes. (I also found a bug with looking up sources for built-in packages, #30, but the feature should be useful regardless.)

A parser cannot infer argument type except for trivial literals, so that won't happen even if the Snail parser gets a major upgrade.

[kskyten]

I think literal types would be sufficient, you can narrow it down further interactively.

[gcv]

We've substantially revised the Julia parsing story. See #33 and 34498596550b0f81. If there are specific further enhancements you'd like to see, please open separate tickets.