Add syntax highlighters for shading languages

[SaschaWillems]

In some spots like the HLSL chapter of the Vulkan guide we have GLSL and HLSL code snippets. It would be nice if we could get syntax highlighting added for those too (which is already the case for C/C++ code snippets).

[oddhack]

I'm not sure what our options are - asciidoctor relies on external code formatters and there are different ones between the Ruby and JS implementations as well as what appear to be occasional changes in what asciidoctor recommends people to use - but we can look into it. It will come down to whether someone has written a plugin for these languages for the specific formatter being used by asciidoctorjs.

[SaschaWillems]

It looks like Asciidoctor also supports highlight.js, which is pretty common: https://docs.asciidoctor.org/asciidoctor/latest/syntax-highlighting/highlightjs/

hightlight.js supports e.g. glsl out of the box. For hlsl we would need to include a custom highlighter based on highlight.js: https://github.com/highlightjs/highlightjs-hlsl

Did that for a branch of Vulkan Tutorial, so if you want I can take a look at it.

[oddhack]

AFAICT antora requires use of highlight.js but to modify it we're going to have to get into the guts of the UI package, which is not AFAIK included amongst the JS modules making up Antora itself (and packaged in this repo). We will probably have to fork the UI anyway to make any meaningful changes to the navigation structure or styles. Also I need to update the packaged Antora in this repo, which is ca. 6 months out of date. I am afraid of versioning issues if everyone just pulls their own modules, and Antora seems to recommend doing it this way, too, though shipping around 90 MB of JS modules inside the repository seems a bit excessive.

[oddhack]

@SaschaWillems I setup a fork of the Antora UI in https://github.com/KhronosGroup/antora-ui-khronos and if you are able to take a look at the highlight GLSL/HLSL integration that would be great. Superficial examination suggests that src/js/vendor/highlight.bundle.js may need to have those languages registered, but that's just a guess.

N.b. to make this have any effect on the demo site I have to rebuild the UI bundle in that repo, then copy it over here, then rebuild the site and push it to the webserver, not at all automatic ATM. So debugging might get pretty tedious.

[oddhack]

@SaschaWillems I setup a fork of the Antora UI in https://github.com/KhronosGroup/antora-ui-khronos and if you are able to take a look at the highlight GLSL/HLSL integration that would be great. Superficial examination suggests that src/js/vendor/highlight.bundle.js may need to have those languages registered, but that's just a guess.

N.b. to make this have any effect on the demo site I have to rebuild the UI bundle in that repo, then copy it over here, then rebuild the site and push it to the webserver, not at all automatic ATM. So debugging might get pretty tedious.

I've added the glsl source highlighter - you can see it in the SPIR-V sample code in https://registry.khronos.org/vulkan/site/spec/latest/chapters/pipelines.html#VkSpecializationMapEntry (which is tagged as glsl source - probably because nobody has defined a SPIR-V grammar for the various source highlighters we use in the spec build).

Incorporating the hlsl grammar will be a bit more involved since it's in a separate JS package from highlight.js itself, but looks manageable.

[SaschaWillems]

Thanks. I gave this a shot some time ago. and adding in the HLSL grammar is indeed a bit more involved. It actually seems to be easier if we do our own highlighting, which is easier to integrate.

[oddhack]

Do you know if the HLSL grammar is available as a node package? That seems like an easier way of integrating it with the site UI but I don't see any references to that on the github site.

[oddhack]

I think once I figure out how to add the highlight grammar .js to the Antora UI and add references to it from their highlight configuration script, it should just work. But it's pretty opaque because all the JS in the default UI gets sent through a minification / bundling / obscuration filter before being packaged for Antora, so I'm asking to try and get clarity on how to properly require() the new grammar from inside that highlighter setup.

Alternatively the HLSL grammar could be published as a node module and added to the UI setup in which case I think it would be easier to configure, but the github project seems pretty dead so we might have to create our own module instead.

[oddhack]

Making some progress. I can get hlsl.js into the Antora UI build now, but it is getting a ton of eslint errors on the file (and also on the minified version). Unfortunately I don't think we can get away with including this highlighter without making changes to upstream.

[SaschaWillems]

As an alternative we could write our own highlighter. Doing this doesn't seem to complicated (gave it a try on a local version of this repo), and in that process we could also add newer and Vulkan related HLSL tokens to the highlighter.

[oddhack]

I believe this is working now - see https://registry.khronos.org/vulkan/site/guide/latest/hlsl.html#_hlsl . That might be the only example of HLSL in the current documents in the site build but there is some highlighting going on. Had to convert hlsl.js to CJS to get it to fit into the Antora UI bundle.

Please close if you agree this fixes the issue.

[oddhack]

As an alternative we could write our own highlighter. Doing this doesn't seem to complicated (gave it a try on a local version of this repo), and in that process we could also add newer and Vulkan related HLSL tokens to the highlighter.

I'm fine with updating src/js/vendor/hlsl.js however seems appropriate. The file is so tiny it's hardly worth trying to synchronize with upstream, which looks pretty dead in any event. Speaking of which, is "POSITIONT" actually an HLSL keyword, or a typo in the upstream?

[oddhack]

Also, if you want to propose a PR here, you can test by 'gulp preview' and editing HLSL source into preview-src/index.adoc (as I've already done for a minimal test).

[SaschaWillems]

Thank you very much for adding this. Highlighting works fine and looks great 👍

Speaking of which, is "POSITIONT" actually an HLSL keyword, or a typo in the upstream?

Indeed looks like a typo. The position keywords (e.g. POSITION0) are properly highlighted though. But in general as you said that highlighter is a bit dated, and also doesn't know about any of the Vulkan keywords or newer HLSL stuff. With it now being part of this project I'll take a look and try to add in new stuff.