jgm / skylighting

A Haskell syntax highlighting library with tokenizers derived from KDE syntax highlighting descriptions
190 stars 61 forks source link

wrong highlighting for TeX command #43

Closed ousia closed 1 year ago

ousia commented 6 years ago

@jgm, as suggested by you, this comes from jgm/pandoc#4466.

For some strange reason, latest pandoc parses:

\def\DesiredFont{TeX Gyre Heros}

into

<div class="sourceCode" id="cb1">
  <pre class="sourceCode tex">
    <code class="sourceCode latex">
      <a class="sourceLine" id="cb1-1" data-line-number="1">
        <span class="fu">\def</span>\DesiredFont{TeX Gyre Heros}
      </a>
    </code>
  </pre>
</div>

It detects \def as command, but it doesn’t \DesiredFont as a command.

In ConTeXt, command pattern matching for highlighting is a backslash followed by any letter ([A-Za-z]), plus any of the characters @!?_ or a character from the last three lines in the code bellow.

local cont = lpeg.R("\128\191")   -- continuation byte

command =  P("\\") * (lpeg.R("az", "AZ") + S("@!?_")   
           + (lpeg.R("\194\223") * cont)
           + (lpeg.R("\224\239") * cont * cont) 
          + (lpeg.R("\240\244") * cont * cont * cont))

To avoid \n highlighted as a command, the following line is introduced:

local anything         = P(1)
local newline          = P("\r") * (P("\n") + P(true)) + P("\n")

not_a_command = P("\\") * (anything - newline)^-1 

Since I cannot code, I adapted the lines above from the ConTeXt source with the explanation Parsing Expression Grammars For Lua. Sorry, but I’m afraid this is the clearest way I can describe it.

Would it be possible to have a similar approach in skylighting for this particular case?

Many thanks for your help.

jgm commented 5 years ago

I've confirmed that Kate behaves the same way. We emulate Kate's highlighting and use the KDE syntax definitions. Perhaps this can be fixed upstream.

jgm commented 1 year ago

fixed upstream and in skylighting