isker / neogrok

Neogrok is a frontend for zoekt, a fast and scalable code search engine.
https://neogrok-demo-web.fly.dev
MIT License
44 stars 3 forks source link

Add syntax highlighting with Shikiji #17

Closed isker closed 10 months ago

isker commented 10 months ago

At long last, we have syntax highlighting with acceptable tradeoffs.

Shikiji is intended to be the next version of the longstanding Shiki. It's fundamentally better in that it simply uses ESM and dynamic imports to break out the wasm blob and the many language grammars into their own JS assets, instead of an ad-hoc asset fetching meechanism. This means it does the right thing when integrating with SvelteKit's vite build tooling, for free.

I have had to make minor patches to Shikiji to get it to have the exact APIs that I want. I could have used the stock ones but they would result in a jankier (and slightly less performant) integration. I will propose upstreaming these changes, but I doubt they will all be acceptable.

The patches required a switch to yarn. I was long getting tired of npm's general jank, but it simply lacks this feature entirely.

The exact approach to highlighting is discussed in the new LineGroup component. The data returned by content-parser in the API also had to change to more easily merge with Shikiji's highlights.

Closes #3.