tree-sitter / tree-sitter-javascript

Javascript grammar for tree-sitter
MIT License
314 stars 108 forks source link

Expose template string chars as nodes? #275

Closed helixbass closed 3 months ago

helixbass commented 6 months ago

It seems unexpected that if you traverse the "leaf" nodes of an AST with a template literal (that has some text content) there are no leaf nodes that contain that text content

For example for the source code `abc` the template_string node has child nodes for `/` but not abc

This appears to be asymmetrical wrt "regular" strings, which surface their contents as string_fragment's

(and more generally I guess it violates some vague "invariant"/expectation I have that a node should either be a "leaf" node (ie have no child nodes) or else all of its non-whitespace characters should be encapsulated by some child node, no idea whether this is actually "a thing" or not)

My use case is that I basically want to compare whether two template strings are "identical" (as far as content/AST structure, so eg `abc${d}efg` and `abc${ d }efg` should be considered "identical" even though they're not source-text identical) and this would presumably be way easier if the text content (here, abc/efg) were exposed as nodes (then I could just do leaf-node-by-leaf-node comparison)

I will open a PR that basically attempts to fix this by changing _template_chars -> template_chars, but I think I ran into something with highlighting/injection that makes it impossible to maintain existing behavior while adding this change

helixbass commented 6 months ago

Actually I think the highlighting/injection issue is possibly resolved "enough", so #276 should be reviewable/potentially mergeable