Highlighting for typst

[nandac]

@jgm I would like to get Syntax Highlighting for Typst code in Pandoc and possibly the Kate text editor.

Are the currently available XML files in skylighting-core generated automatically or are they hand-crafted?

I would like to contribute an XML for Typst but getting a full list of keywords and creating the XML file by hand seems cumbersome.

What would be the best approach for me to go about this?

[jgm]

See the README. The xml files come from upstream (KDE's syntax highlighting framework). The README contains instructions about where to look for documentation, how to submit a new syntax definition to KDE, etc.

As for generating a list of keywords, I don't have specific suggestions. You could try scraping from the typst website, or directly from their source code.

[nandac]

Thanks @jgm I will look into how I may generate a list of keywords.

From what I understand it would be best for me to contribute the syntax file to KDE and then have Pandoc pick them up sometime in the future which will also give Pandoc Typst highlighting support.

[jgm]

Correct!

[nishanthkarthik]

I wrote this for very basic highlighting in kate, which suffices for my needs. I don't think it's mature enough to contribute upstream yet.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE language SYSTEM "https://raw.githubusercontent.com/KDE/syntax-highlighting/master/data/schema/language.xsd">

<language name="Typst" section="Sources" version="1" kateversion="5.0" indenter="cstyle"
          extensions="*.typ" mimetype="text/x-typ" author="nishanthkarthik" license="MIT">
    <highlighting>
        <list name="math-symbols">
            <item>tack</item>
        </list>

        <contexts>
            <context name="Normal" attribute="Normal">
                <DetectChar char='"' context="str"/>
                <DetectChar char="#" context="decl"/>
                <DetectChar char="$" context="math" lookAhead="true"/>
                <RegExpr String="^(=)+" context="heading"/>
                <RegExpr String="^\s*[+-]" attribute="Bullet"/>
                <keyword String="math-symbols" attribute="MathSyms"/>
                <RegExpr String="\*[A-Za-z0-9 ]+?\*" attribute="Bold"/>
                <RegExpr String="_[A-Za-z0-9 ]+?_" attribute="Italic"/>
                <RegExpr String="`[^`]+?`" attribute="Code"/>
                <RegExpr String="```.+?```" attribute="Code"/>
            </context>

            <context name="str" attribute="String" lineEndContext="#stay">
                <RegExpr String='[^"]+' attribute="String" context="#stay"/>
                <DetectChar char='"' attribute="String" context="#pop"/>
            </context>

            <context name="decl" attribute="Decl" lineEndContext="#pop">
                <DetectIdentifier context="#stay" attribute="Decl"/>
                <RegExpr String="[^A-Za-z0-9]" context="#pop" lookAhead="true"/>
            </context>

            <context attribute="Math" name="math" lineEndContext="#stay">
                <DetectChar char="$" context="#pop" attribute="Math"/>
            </context>

            <context attribute="Heading" name="heading" lineEndContext="#pop">
                <IncludeRules context="Normal"/>
            </context>
        </contexts>

        <itemDatas>
            <itemData name="Normal" defStyleNum="dsNormal"/>
            <itemData name="Italic" defStyleNum="dsNormal" italic="true"/>
            <itemData name="Bold" defStyleNum="dsNormal" bold="true"/>
            <itemData name="Code" defStyleNum="dsAnnotation"/>
            <itemData name="Bullet" defStyleNum="dsRegionMarker" bold="true"/>
            <itemData name="String" defStyleNum="dsString"/>
            <itemData name="Decl" defStyleNum="dsFunction"/>
            <itemData name="Math" defStyleNum="dsSpecialChar"/>
            <itemData name="MathSyms" defStyleNum="dsBuiltIn"/>
            <itemData name="Heading" defStyleNum="dsNormal" bold="true" underline="true"/>
        </itemDatas>
    </highlighting>
</language>

[nandac]

@jgm @nishanthkarthik I have also been working on his in the intervening months since my first message:

<?xml version="1.0" encoding="UTF-8"?>
<language name="Typst" section="Markup" extensions="*.typ" version="1.0" kateversion="5.0"
    mimetype="text/x-typst" author="Nandakumar Chandrasekhar" license="MIT" style="typst"
    indenter="cstyle" priority="1" casesensitive="1">
    <highlighting>
        <list name="constants">
            <item>none</item>
            <item>auto</item>
            <item>true</item>
            <item>false</item>
        </list>
        <list name="other">
            <item>let</item>
            <item>as</item>
            <item>in</item>
            <item>set</item>
            <item>show</item>
            <item>#let</item>
            <item>#as</item>
            <item>#in</item>
            <item>#set</item>
            <item>#show</item>
        </list>
        <list name="operator">
            <item>and</item>
            <item>or</item>
            <item>not</item>
        </list>
        <list name="control">
            <item>if</item>
            <item>else</item>
            <item>for</item>
            <item>while</item>
            <item>break</item>
            <item>continue</item>
            <item>return</item>
            <item>#if</item>
            <item>#else</item>
            <item>#for</item>
            <item>#while</item>
            <item>#break</item>
            <item>#continue</item>
            <item>#return</item>
        </list>
        <list name="import">
            <item>import</item>
            <item>include</item>
            <item>export</item>
            <item>#import</item>
            <item>#include</item>
            <item>#export</item>
        </list>
        <contexts>
            <context attribute="Normal Text" lineEndContext="#pop" name="Normal Text">
                <!-- Constants -->
                <keyword attribute="Constant" context="#stay" String="constants" />
                <!-- Length -->
                <RegExpr attribute="Constant" context="#stay"
                    String="(\b(\d*)?\.?\d+([eE][+-]?\d+)?(mm|pt|cm|in|em)\b)" />
                <!-- Angle -->
                <RegExpr attribute="Constant" context="#stay"
                    String="(\b(\d*)?\.?\d+([eE][+-]?\d+)?(rad|deg)\b)" />
                <!-- Percentage -->
                <RegExpr attribute="Constant" context="#stay"
                    String="(\b(\d*)?\.?\d+([eE][+-]?\d+)?%)" />
                <!-- Numeric Fraction -->
                <RegExpr attribute="Constant" context="#stay"
                    String="(\b(\d*)?\.?\d+([eE][+-]?\d+)?fr)" />
                <!-- Numeric Integer -->
                <RegExpr attribute="Constant" context="#stay"
                    String="(\b(0x[0-9a-zA-Z]+|(0b|0o)?\d+)\b)" />
                <!-- Numeric Float -->
                <RegExpr attribute="Constant" context="#stay"
                    String="(\b(\d*)?\.?\d+([eE][+-]?\d+)?\b)" />

                <!-- Keywords -->
                <keyword attribute="Keyword" context="#stay" String="other" />
                <keyword attribute="Keyword" context="#stay" String="operator" />
                <keyword attribute="Keyword" context="#stay" String="control" />
                <keyword attribute="Keyword" context="#stay" String="import" />

                <!-- Function -->
                <RegExpr attribute="Function" context="#stay" String="(\b[[:alpha:]_][[:alnum:]_-]*!?(?=\[|\())" />
                <RegExpr attribute="Function" context="#stay" String="(#[[:alpha:]_][[:alnum:]_-]*!?(?=\[|\())" />

                <!-- Variables -->
                <RegExpr attribute="Variable" context="#stay" String="(\b[[:alpha:]_][[:alnum:]_-]*(?=:))" />
                <RegExpr attribute="Variable" context="#stay" String="(#[[:alpha:]_][.[:alnum:]_-]*)" />
                <RegExpr attribute="Variable" context="#stay" String="([[:alpha:]_][.[:alnum:]_-]*)" />
                <RegExpr attribute="Variable" context="#stay" String="(\b[[:alpha:]_][[:alnum:]_-]*\b)" /> -->

                <!-- String Start -->
                <DetectChar attribute="String" context="string" char="&quot;" />

                <!-- Single Line Comment Start -->
                <Detect2Chars char="/" char1="/" attribute="Comment" context="SingleLineComment" />
                <!-- Multi Line Comment Start -->
                <Detect2Chars char="/" char1="*" attribute="Comment" context="MultiLineComment" beginRegion="Comment" />
            </context>
            <context attribute="String" lineEndContext="#stay" name="string">
                <!-- String End -->
                <DetectChar attribute="String" context="#pop" char="&quot;" />
            </context>
            <context attribute="Comment" lineEndContext="#pop" name="SingleLineComment">
                <!-- Single Line Comment End -->
                <DetectSpaces />
                <IncludeRules context="##Comments" />
            </context>
            <context attribute="Comment" lineEndContext="#stay" name="MultiLineComment">
                <!-- Multi Line Comment End-->
                <DetectSpaces />
                <Detect2Chars char="/" char1="*" attribute="Comment" context="MultiLineComment"
                    beginRegion="Comment" />
                <Detect2Chars char="*" char1="/" attribute="Comment" context="#pop"
                    endRegion="Comment" />
                <IncludeRules context="##Comments" />
            </context>
        </contexts>
        <itemDatas>
            <itemData name="Normal Text" defStyleNum="dsNormal" />
            <itemData name="Function" defStyleNum="dsBuiltIn" />
            <itemData name="Keyword" defStyleNum="dsKeyword" />
            <itemData name="Constant" defStyleNum="dsConstant" />
            <itemData name="Comment" defStyleNum="dsComment" />
            <itemData name="String" defStyleNum="dsString" />
            <itemData name="Variable" defStyleNum="dsVariable"/>
        </itemDatas>
        <general>
            <comments>
                <comment name="singleLine" start="//" />
                <comment name="multiLine" start="/*" end="*/" region="CommentFolding" />
            </comments>
            <keywords casesensitive="1" />
            <folding indentationsensitive="0" />
            <emptyLines>
                <emptyLine regexpr="\s+" />
                <emptyLine regexpr="\s*#.*" />
            </emptyLines>
            <spellchecking>
                <encoding char="á" string="\'a" />
                <encoding char="à" string="\`a" />
            </spellchecking>
        </general>
    </highlighting>
</language>

The only problem with it is that it highlights keywords like and, or, if etc in the plain text which I have not been able to fix. I would like to collaborate with someone so that we have something ready for prime time.

[nishanthkarthik]

I'm currently blocked by highlighting chunks like *_text_* or _ foo *bar* foo_. Meanwhile, I'm also looking at understanding how it works upstream in vscode https://github.com/typst/typst/blob/main/tools/support/typst.tmLanguage.json

Creating folding regions for heading levels = is also a pain. The way markdown.xml implements it is hacky.

[nandac]

@nishanthkarthik I think I have some code for that on a later version of my XML which is not fully tested. I will dig it up and get back to you with the snippet.

Can you please connect with me on LinkedIn so that we can collaborate: https://www.linkedin.com/in/nandakumar-chandrasekhar-a400b45b/

[nishanthkarthik]

I don't use linkedin much but if you have a github repo for the project, I'd be happy to help!

[nandac]

@nishanthkarthik I do not have a GitHub repo yet. I think it might be a good idea to fork Kate's syntax highlighting repository so that we can submit a pull-request when we are ready.

Let me look into this and get back.

[nandac]

@nishanthkarthik I think KDE has its set-up of GitHub at https://invent.kde.org/ where you need to create an account. I created an account and forked the repository: https://invent.kde.org/nandac/syntax-highlighting

Let me know if you can view it and clone it.

[silby]

available upstream https://invent.kde.org/frameworks/syntax-highlighting/-/blob/master/data/syntax/typst.xml?ref_type=heads