column_header when tilde is not preceded by space

[lervag]

The Tree-sitter version of the help filetype ~does not currently support the "column heading" item~ has a known limitation for handling the column header item. This item is documented in :help help-writing just before :help help-codeblock:

To define a column heading, use a tilde character at the end of the line.
This will highlight the column heading in a different color.  E.g.

Column heading 

(Btw.: I actually believe there may be a slight bug in the help file here, as I think the example should contain the tilde character.)

If the tilde does not have a preceding space, the current parser does not recognize it. So, the following syntax with old style syntax rules

Heading here~
Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy
eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam
voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita
kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

Should look something like this (depending on color scheme and so on):

Edit: I'm sorry, I just realized this was mentioned as a current limitation here. So, if I add a preceding space to the tilde, then it will be parsed correctly. Perhaps this issue then can remain open for handling the case without preceding space? I've reworded it to make it more clear.

[clason]

(Btw.: I actually believe there may be a slight bug in the help file here, as I think the example should contain the tilde character.)

Vimhelp is woefully underspecified... The only full documentation is syntax/help.vim.

On the other hand, this parser is opinionated: If there are multiple valid ways, it will in general only try to support one.

In this case, it (for technical reasons) requires a space before the tilde, so the following does work:

Heading here ~
Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy
eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam
voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita
kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

[clason]

Expanding on the technical reason: As this parser does not use an external scanner, it relies on whitespace for separating tokens, so this limitation is by design.

[lervag]

In the README, it says:

- `column_heading` currently only recognizes tilde `~` preceded by space (i.e.
  `foo ~` not `foo~`). This covers 99% of :help files.

The "currently" seems to indicate that the limitation may be relaxed in the future. I would propose to either

1. Address this issue and resolve the limitation, or
2. Specify more clearly that the limitation remains (for technical reasons) and remove the "currently". This issue may then be resolved as "won't fix".

[clason]

I can't speak for @justinmk's plans, but as far as I can tell, this is not a limitation that can be addressed without a massive redesign -- it's not just a 't' that was forgotten to be crossed...

At least from the POV of Neovim itself, the official policy in such cases is to simply change the help files to conform to the stricter spec. This is also the very first sentence of the README...

[lervag]

Thanks; I'll wait for @justinmk to reply. I don't mind adhering to stricter conventions, of course. But the "currently" in the README makes it sound like this is going to change in the future. If it (most likely) isn't, then it would be easier as a plugin author to adjust accordingly :)

(I probably will now, anyway, but still.)

[clason]

One of my secret hopes is that this parser will eventually serve as an actual, complete, and unambiguous specification for the "help grammar" to make it easier to write -- and generate from other formats like docstrings or Markdown...

[justinmk]

Not really interested in litigating the presence of "currently", but the readme also mentions:

The first step should always be to try to fix the input rather than insist on a grammar that handles vimdoc's endless quirks.

Anyhoo, improvements are welcome if they don't lead to insane complexity, but otherwise this is likely to remain a "known issue".

[lervag]

Thanks; I'll update help files I'm responsible for accordingly.

[clason]

One step closer to enabling tree-sitter highlighting for help files by default :)

[lervag]

Hehe, I actually already did. Which explains why I opened this issue in the first place; the implication being that I found the tree-sitter highlighting now solid enough to scratch the legacy version.