Syntax highlighting issues with template string literals in JavaScript and TypeScript

[eeshugerman]

I'm not sure if this is the right repo for this issue, so feel free to point me elsewhere.

const foo = 1;

`bar`; "bar"; // ok, just for reference

foo;
`${foo} bar`; // "foo" is the wrong color

foo.toString;
`${foo.toString} bar`; // "foo" is the wrong color (rest is ok)

1;
`${1} bar`; // ok

1 + 1 === 1;
`${1 + 1 === 1} bar`; // operators are the wrong color

1 + new Date();
`${1 + new Date()} bar`; // "new" is the wrong color

Also (but of less importance than the issues above) it would be nice if $, {, and } did not get the string color (like how GitHub does it), since they're not actually part of the string.

[ubolonton]

It's an issue with the highlighting queries in tree-sitter-langs, probably here in javascript/highlights.scm.

You can try editing that file in your tree-sitter-langs directory and then running revert-buffer in the typescript buffer.

[eeshugerman]

Thanks for the pointers! I found that the issues are mostly fixed if I remove template_string from this line so that it's just (string) @string. (EDIT: Actually this just does what it seem like it would do -- doesn't color template strings as strings -- except on js2-mode. It seems non-tree-sitter highlighting is not completely overridden with js2-mode somehow.)

As for the remaining issues, from here it gets weird: the syntax highlighting actually changes slightly when I edit a buffer (e.g just adding newline). (I didn't notice this before, but it happens both with and without the fix described above.) The syntax highlighting is slightly off both before and after I edit the buffer, but in different ways.

Here's a video (don't mind the new color theme): https://user-images.githubusercontent.com/25518211/144759507-d58b8f05-90a9-4695-b4ad-bd84aea766d3.mp4

I'm not seeing this with emacs -Q, so I suppose it's another package interfering. Any chance you've seen something like this before?

EDIT: Seems the "syntax highlighting changes when I edit the buffer" thing is just with js2-mode. Not seeing it with javascript-mode or typescript-mode. I'll put that on the back burner for now and keep debugging tree-sitter highlighting with one of the latter.

[eeshugerman]

OK I think I got it worked out, please see the ~draft~ PR above. ~I left it as a draft for now because of the question about variable references, and also I'm not sure if this is the best way to fix the issue.~