Proposal: Grand Docs Migration

[novusnota]

Goals

• Easy i18n (translations of the docs from English to other languages) without convoluted build systems

• Easy versioning of the docs: the separation into the latest (for users) and the upstream/dev versions via folders. Upon any releases, we'll have to sync the dev versions with the latest one.

• Continue using LaTeX, React Components, full website search, as well as TextMate grammars for code block highlighting. Keeping highlighting for inline code segments would be nice too.

• Continue using SSG (static site generation) and free deploy to Github Pages

• Improve the SEO — as of now, individual pages don't stand out from each other when linked in chats or discovered in search engines

Other options considered

Docusaurus:

• Popular, has a lot of things built for it and a large community. Has stable releases.
• Very bulky and rather unresponsive.
• Things are tough to spot — the pagefind plugin docs are missing, and search setup is, apparently, a hard problem

VuePress:

• Rather popular, has lots of Vue integrations
• React components are nowhere to be found, other means of extensibility are very Vue-locked (expectedly)
• Plugin API is accessible, but rather limiting compared to Docusaurus and Starlight, so custom extensions are less likely to happen.

Solution

Starlight — an Astro-powered documentation framework, that allows for great extensibility and bundles (among other things): advanced i18n support via i18next, site navigation, search, SEO, easy-to-read typography, code highlighting, dark mode and more. Also, it's very lightweight.

Extra pros discovered:

• Aside from the full-text search (Ctrl-K), the user can simply search by the contents (Ctrl-F) and discover titles of pages throughout the documentation sections. That is because Book, Cookbook, Reference and Ecosystem are now gathered together in the sidebar on the left now instead of the buttons on top.
• Every page gets an "Overview" button in table of contents, and every page can have it's SEO-related parameters customized on the fly
• Bringing new functionality throughout the site is very easy, and all while staying in the SSG mode (unlike Nextra)
• Nextra only used the dark theme of the code even in when the whole site theme is set to light. That was a limitation, which was tough to circumvent, and so users of light themes got very bleek color highlighting as a result. Well, not anymore — the new code block highlighting uses the separate dark and light themes as expected and out of the box.

Cons discovered:

• By default, there's no highlighting of inline code blocks and no existing plugins for it yet. Local implementation went rather smooth, wasn't a big deal
• LaTeX and clickable headings with custom links have to be integrated in via rehype and remark plugins. Local integration went quite smooth, so that wasn't a big deal too
• The full-site search may be a bit slower compared to Nextra, but it provides better results. And now, users can also search the contents via Ctrl-f as mentioned above.

About i18n and translations

All the UI elements have built-in translations to a lots of languages, including Chinese and Simplified Chinese, so we don't have to translate those, only the content of the pages.

Regarding that, one path we might take with Starlight and Crowdin (for translations) may look as follows:

1. We use https://store.crowdin.com/i18next-json built-in format, and export files in .json according to i18next-json structure
2. We add those files from time to time over to src/content/i18n/ folder in Starlight docs
3. On the non-English .mdx pages instead of the "raw" text we place calls to Astro.locals.t() and pass the keys from the files from step 2.

The big issue with that approach arises from the volatile nature of Tact docs — texts get changed often and even whole paragraphs can appear and disappear at times. Containing that complexity within the keys in .json files puts WAY too much maintainance burden on the translators. Besides, those are just inconvenient to write since the whole picture of the text would be hidden away, allowing only single phrases or paragraphs to be translated at the time, lacking the larger context of the whole page.

My proposal is simple — just use the .mdx files for the translations. Crowdin already supports the .mdx format out of the box (including YAML or TOML frontmatters, see: https://store.crowdin.com/mdx), so we can:

1. Put .mdx files onto Crowdin for translators
2. Pull translated files from it from time to time (like, once a month or so)
3. Place those files under src/content/docs/zh-cn (for Simplified Chinese) and that's it

Now, regarding sidebar and the translations of titles — nothing extra has to be done there! If we use internal links (i.e. just names of the files, which must stay the same in all localizations), we get the localized titles out of content frontmatters of translated pages for free. See: https://starlight.astro.build/guides/sidebar/#internationalization-with-internal-links.

P.S.: I believe that all technical writers and translators must know Markdown and bits of HTML and YAML to be able to figure MDX out in less than an hour, really. And giving more context (i.e. full file instead of the disjoint set of strings/lines) to translators is ALWAYS a plus.

Steps

• [x] Migrate the existing pages (almost there, only need to replace React Components such as <Callout> where appropriate)
• [x] Convert the index page
• [x] Send a PR integrating the new docs to this repo (#880)
• Gradually deprecate tact-docs repo, while keeping the list of issues stored there for a little while

[anton-trunov]

By default, there's no highlighting of inline code blocks and no existing plugins for it yet. Local implementation went rather smooth, wasn't a big deal

What does it mean "Local implementation went rather smooth"?

[novusnota]

What does it mean "Local implementation went rather smooth"?

I just wrote a small rehype plugin (a single file with <100 lines with comments) and included it into the astro.config.mjs. Tested, debugged, and then achieved the same (theme and all) inline code highlighting, as it's seen in Nextra framework in the current Tact docs.

P.S.: Inline code highlighting means this gets highlighted, not:

this

Which is a code block highlighting :)