sogaiu
commented
5 months ago I looked into tags.scm [1] a bit before, and the conclusion I reached at that point was that due to the way tree-sitter-clojure is constructed (partly because of present tree-sitter limitations / functionality / scope), it didn't seem likely that an all that meaningful tags.scm file can be created...but perhaps it depends on the details of how it's to be applied.
To explain a bit, taking a look at one of the pages referenced in the issue you linked to, one can see:
Class definitions @definition.class
Function definitions @definition.function
Interface definitions @definition.interface
Method definitions @definition.method
Module definitions @definition.module
Function/method calls @reference.call
Class reference @reference.class
Interface implementation @reference.implementationUnlike most other tree-sitter grammars, tree-sitter-clojure (and a number of other comparable things for lisp-likes) doesn't yield parse trees with "higher level" constructs (e.g. function definitions). See here for background and rationale.
The only types of things that end up in a parse tree are things like lists, vectors, and other "low level" constructs. Even function definitions are not represented in the tree (see link above for a discussion of that particular point, but it's an example of a more general issue).
I'm not sure how aider uses tags.scm files. Perhaps if that was clearer, one could make a better assessment. This directory at the aider repository contains *-tags.scm files for a variety of languages. If you take at the one for elisp:
;; defun/defsubst
(function_definition name: (symbol) @name.definition.function) @definition.function
;; Treat macros as function definitions for the sake of TAGS.
(macro_definition name: (symbol) @name.definition.function) @definition.function
;; Match function calls
(list (symbol) @name.reference.function) @reference.functionI think this makes it clear that, for at least tree-sitter-elisp, there is a reliance on the grammar providing some ability to recognize definitions (e.g. IIUC function_definition and macro_defintion are names of nodes that the corresponding grammar recognizes) and calls.
The third item (the one that is like (list ...)) hints that it might be of some use to "fake" recognition of function calls by picking out (something like) "lists that start with symbols". That will match other things though, including macro calls as well as well a special form is used. I don't know if that makes a difference for accuracy in the context of aider.
I think it could be useful to try to ascertain how well aider works with Elisp (may be aider folks can be asked or you can test directly?). If it doesn't work that well, I would suspect that it's not going to work better for Clojure (since tree-sitter-clojure lacks things like function_definition, it seems likely to me that results are unlikely to ever be better than those for elisp -- just a guess though), though I suppose one may not know until something is tried.
If you still want to try get aider working with Clojure, the following are some things that might help (note: I suggest browsing the following items and thinking over an initial plan, mix and match, change ordering, etc. (you can revisit / revise later)):
- Scan this part of the official tree-sitter docs. It's a lot to take in (and I wouldn't try to really understand it in detail on a single reading), but I think it's good to know what kinds of docs exist. The kind of information you'll likely want to be able to make sense of has to do with what's in a
grammar.jsfile and what the bits mean. It also has some background which will likely turn out to be useful when working with tree-sitter. - Clone Wilfred's tree-sitter-elisp repository. According to this aider document, that's the elisp grammar being used (there is actually another one in existence that is a fork with some changes). Then start looking over the contained
grammar.js. I think you'll findlist,function_definitionandmacro_defintionmentioned. Try to understand the associated definitions. These might not directly help much for Clojure's case, but it might be a good focused initial task for starting to become familiar with typicalgrammar.jsfiles -- and Emacs Lisp is much closer to Clojure than many other programming languages. - Try to understand how the content of
tree-sitter-elisp-tags.scm(the whole content is considered a single query in tree-sitter lingo -- a bit confusing, but that's how it is). For this, I suggest becoming familiar with at least at this section of the official docs. It may be that learning interactively with the tree-sitter playground could be helpful, but that version doesn't have any lisp-likes. If you install thetree-sittercli locally and have the appropriate setup, you can run a local version of the playground for specific grammars, but this may be a fair bit of work depending on your background, motivation, energy, etc. This is because to get that plaground running you may need to build an appropriate.wasmfile and that has some finicky bits. This page might have some useful information regarding this. Note that thetree-sittercli program has a subcommand namedquerywhich can be used to test out queries -- the playground can be nice (especially initially), but it's not strictly necessary. - Figure out some way to test / experiment with
tree-sitter-elisp-tags.scm. I don't know anything about aider, but if you can run it locally, that would likely be helpful because you can iterate over different versions of your file. Running locally is not necessarily essential, what is important is to be able to try different versions of the file in a manner that doesn't take too much time to see the effects of changes you make. There might be suggestions about how to do something appropriate at the aider repository. This document looks like it has a couple of relevant looking items. - Once you are comfortable with tweaking
tree-sitter-elisp-tags.scm, try to sketch out what might be necessary for Clojure's case and test with that. At a minimum, creating atree-sitter-clojure-tags.scmfile that contains something similar to the(list ...)bit (this will be for idenitfying "calls" / "references") discussed earlier seems like it could be a good starting point, though I suspect it will be necessary to be able to express an expression that helps to identify definitions as well. The node names are different in tree-sitter-clojure (from those in tree-sitter-elisp), so you'll likely want to check thegrammar.jsfile in this repository (e.g. it's notlisthere, but ratherlist_lit). However, there might be other necessary bits. For example, it looks like aider depends on py-tree-sitter-languages, which doesn't appear to list Clojure. It may be that getting support for tree-sitter-clojure added there is necessary (at least eventually if aider uses py-tree-sitter-languages). In the near term, it may be enough to create a fork that includes Clojure support and get whatever version of aider you are testing with to use that instead. There might be other things that need to be changed, but may be the aider folks can help identify those things. - Consider joining the tree-sitter discord (see the top of this page) if you haven't already. There are people there who might be able to help you out from a general tree-sitter perspective. I don't know if there are any support channels for aider (apart from its issues), but it seems worth checking.
May be there is something of use in the above (^^;
I'm ok to discuss this further if you wish -- not sure how much and for how long, but we'll see as it goes.
[1] It's tags.scm btw, not tags.csm :)
hewcaw
I'm trying to add Clojure/CLJS support for https://github.com/paul-gauthier/aider/issues/373 and it requires this file. I don't know anything about Tree-sitter but do really want to help, so if you have some free time to bare it would be really helpful if you could provide some about instruction / direction, thank you!