Weird highlights due to children of (code) node

[taekwombo]

Hi, some time ago I have noticed a problem where some tokens from injections in vimdoc code block are highlighted with color of @text.literal.vimdoc.

nvim -V:

NVIM v0.10.0-dev-cdc8bac
Build type: RelWithDebInfo
LuaJIT 2.1.1700008891
Run "nvim -V1 -v" for more info

nvim -u NORC:

:Inspect over age variable (its color is the problem).

:InspectTree for the code block:

 (line) ; [6:1 - 10:0]
  (codeblock) ; [6:1 - 10:0]
   (language) ; [6:2 - 7:0]
   (code) ; [7:1 - 10:0]
    (comment) ; [7:5 - 76]
    (declaration) ; [8:5 - 12]
     type: (primitive_type) ; [8:5 - 7]
     declarator: (identifier) ; [8:9 - 11]
    (declaration) ; [9:5 - 30]
     (storage_class_specifier) ; [9:5 - 10]
     (type_qualifier) ; [9:12 - 16]
     type: (primitive_type) ; [9:18 - 20]
     declarator: (init_declarator) ; [9:22 - 29]
      declarator: (identifier) ; [9:22 - 24]
      value: (number_literal) ; [9:28 - 29]
    (line) ; [7:1 - 8:0]
    (line) ; [8:1 - 9:0]
    (line) ; [9:1 - 10:0]

Regular C file

---

So, my first guess was that (line) nodes somehow break the highlights. I updated the parser a little bit so that the (line) children of (code) are inlined and this is the result:

There was the same issue in https://github.com/nvim-treesitter/nvim-treesitter/issues/4887 - they fixed it by lowering the priority of the code block highlight.

Here is my approach (it will require snapshot updates). Let me know if I should create a PR for this issue and which way of fixing it you prefer.

From e4b7f10306b6fa4080e8532acb5e8bcc6c08fadf Mon Sep 17 00:00:00 2001
From: taekwombo <88032359+taekwombo@users.noreply.github.com>
Date: Sat, 18 Nov 2023 19:05:37 +0100
Subject: [PATCH] fix: inline "code" descendants

---
 grammar.js | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/grammar.js b/grammar.js
index aff66e2..85e9372 100644
--- a/grammar.js
+++ b/grammar.js
@@ -13,6 +13,7 @@ const _li_token = /[-•][ ]+/;
 module.exports = grammar({
   name: 'vimdoc',

+  inline: ($) => [$.line_code],
   extras: () => [/[\t ]/],

   // inline: ($) => [
@@ -125,7 +126,7 @@ module.exports = grammar({
       choice(
         alias(token.immediate(/[a-z0-9]+\n/), $.language),
         token.immediate('\n')),
-      alias(repeat1(alias($.line_code, $.line)), $.code),
+      alias(repeat1($.line_code), $.code),
       // Codeblock ends if a line starts with non-whitespace.
       // Terminating "<" is consumed in other rules.
     )),
-- 
2.42.0

[justinmk]

Your patch looks fine, thanks! PR welcome, hopefully with a test case :)

What does the tree look like after your change?

[taekwombo]

I was mistaken, there is no issue with the grammar structure here. I have spent some time to check this issue again, this time I wiped out both old runtime files of nvim and any plugins then installed nvim from source again.

Regardless of the changes there I made to the grammar there was no difference in highlighting. :sweat_smile:

If I understand what happened correctly, it goes like this:

• all contents of codeblock are highlighted with @text.literal - this is correct
• injected parser parses the contents of the codeblock and highlights some nodes accordingly to its highlights queries - this is also correct
• nodes that are not highlighted by injected parser will inherit codeblock color, even though in regular file they would be displayed with Normal highlight

If the range handled by injected parser would be highlighted with Normal the final highlights would be the same as in regular file. I will search for issues/discussions about it and try to come up with some solution.

I am closing this issue because this has nothing to do with grammar.

[clason]

Yes, this is the correct interpretation. You can check your colorscheme to make sure it actually assigns colors to "white" (or black, depending on your background) instead of setting them to NONE to default to the built-in Normal group. These should then still paint over the codeblock.

[taekwombo]

Colorscheme is okay - checked that with built-in retrobox. It's more likely the context that matters.

Let's say I have defined only the following highlights queries for C:

((identifier) @variable)

Then I would end up with those highlights in C file: and those highlights in help file:

Injected highlights work fine, but the Normal highlight color for injected highlights becomes whatever it paints on. :)

[clason]

Yes, that's my point -- it shouldn't be Normal.