Move docs into one logical component

[fhennig]

With the harmonization of the Operator versions to conform to the Platform version, all our documentation components are versioned with the same number. This gives us the opportunity to further simplify the docs for the user! What we could do:

Benefits

• A single sidebar navigation for all content - Currently, as every Operator is its own Antora component, it has its own version and its own pages and menu. We could move all pages into one component. This way it would be easier to navigate the page and it is transparent to the reader which pages exist, as they can all be found in the menu.
• A single version dropdown for the whole page - At the moment, every Operator has its own dropdown (due to it being its own omponent). Even with the harmonized versions, this will still be the case. If the user has 22.11 selected in one operator and switches to docs for another, the stable docs for that operator will be selected. It would be better if the same platform version is selected for each operator, to avoid confusion.
• Greatly simplified internal referencing of other pages - At the moment, if we want to link to a concept page from an Operator, we have to link to a different component, which also means we have to explicitly state the version of the component. To do things very correctly, every xref across a component boundary should point to an explicit version. If no version is stated, the link goes to the latest stable. This has the effect of unversioned links in old docs versions linking to possibly inconsistent documentation. This is something we do not do at the moment, because it is very difficult to keep in mind and maintain. With one version and the changes suggested below, we would not be linking across a component boundary anymore, and so any link would always be to the same version automatically.

A preview of what it would look like:

The Trino Operator docs are now in the same navigation bar as the platform level docs, under their own section. Here, only the getting started guide was moved. The version dropdown where nightly is selected affects all docs at the same time, platform level and Trino docs.

How it works

Instead of multiple components, we would move to having one single "Platform" component. This is more in line with how we want to user to think about it too: It all works together, it's versioned together, it's one "thing". Antora components can also be distributed across different repositories. We would change our Operator Antora components to all belong actually to the same distributed component.

If Antora discovers two or more antora.yml files that specify an identical component name and version, it considers all of the files in the subsequent standard directory sets to belong to the same component version, even though the source files were collected from multiple locations.

Having just a single component leads to all the benefits above. But there are also drawbacks, depending on how it is implemented. The differences are mostly in regards to how we deal with existing pre-change versions of operator docs.

Implementation

Using a distributed component for current and future versions is straightforward to implement (I have spiked it in Trino). The docs for 23.1 onward would then live in the single "home" component. The question then becomes, what to do with older Operator versions that use a different versioning scheme.

A: Leave old versions as they are

We could move to a distributed component for future operator versions only, and leave the old, multiple component setup for older versions.

• ✅ Causes no breaks in existing links.
• ✅ No changes to older files required.
• ❌ Navigating to older docs will become confusing
• ❌ The setup gets confusing to maintain (but we can just drop old versions eventually).

B: Change versioning from 22.06, 22.09 and 22.11, drop older versions

We could go and change the docs versions for these older releases to also move them to distributed components, using the platform versions they belong to. For example, for Druid, 0.8 would become 22.11. Any operator versions that do not belong to a platform release would be dropped.

• ✅ Coherent navigation across versions
• ✅ Easier to maintain into the future
• ❌ A lot of work to go back and edit older released docs
• ❌ Some older docs would be deleted (for the non-platform-release operator versions)
• ❌ 22.06 platform version is not actually the operator version at the time. This is difficult to communicate to the reader
• ❌ Existing links to older versions would break

Due to the links being constructed from versions and component IDs, changing existing versions or components will lead to links to break for that component/version. /druid/stable/index.html will become /home/stable/druid/index.html. This is quite severe. I don't know how many people are already linking to docs pages. I don't know how this will affect our Google indexing issues as well. I do know that since almost nothing is indexed yet, we would at least not lose out so much here.

Optional follow up changes if we go for breaking links

If we were to break links, we should consider doing any other URL changes at the same time. There are two changes that I would really like to do:

• rename home component to ROOT to omit it from the URLs https://docs.antora.org/antora/latest/module-name-key/ or rename home to platform or sdp.
• Change the html_extension_style from the default to drop or indexify if we want to do that: [docs] We would have /stable/druid/ instead of `/stable/druid/index.html.

With our new versions but current URL scheme, the link to druid would look like this:

https://docs.stackable.tech/home/23.1/druid/index.html

with the changes applied I suggested above, it would look like this

https://docs.stackable.tech/23.1/druid/

Summary

I propose to move all our documentation into one [https://docs.antora.org/antora/latest/distributed-component-version/](distributed component). This would lead to easier reasoning about the docs, easier linking, a single version for all docs and a single version selector, and having all pages available in a single navigation for the user. Depending on the implementation it might be more or less difficult, and could to link breakages.

[fhennig]

Feedback from the Arch Meeting

• Generally positive sentiment towards the idea
• We do have a slightly growing exposure on Google and it would be good not to mess it up by breaking links
• how to deal with older versions is still a problem
  • Idea: freeze current docs and put them somewhere to get around dealing with it (that would probably still break links?)
  • go with option (A) as above (only adopt the logical component for future releases)
  • We don't get that many clicks on the docs -> just delete all the old versions
• Could we prevent some breakage by using redirects?

Currently I think we should soft-break some links but put in redirects. Would be a good balance between not breaking links completely and not having to maintain too much old structure.

[fhennig]

Antora has some native support for redirects: https://docs.antora.org/antora/latest/page/page-aliases/#page-aliases-and-bulk-url-redirects - However, they explicitly state that this feature should not be used for moving components:

If you remove a large number of pages, such as if you remove or rename a component or component version, and you want to preserve the existing URLs, you should not attempt to do so using page aliases. Page aliases were not designed to provide bulk URL redirects. Instead, we strongly recommended that you use the router provided by your web server or host when you remove or rename a component or component version.

That is unfortunate. Since we use github pages, we don't have any redirect mechanisms available.

We could go against their advice and use the redirects anyways. Or we could move to Netlify, where we also get free hosting and they support proper redirects. Moving from GitHub pages to Netlify shouldn't take long.

---

Edit:

I asked on the Antora Zulip chat about the reason for the recommendation against using bulk page aliases for bulk redirects, and got this response:

You are free to not follow the recommendations. After all, they are just recommendations. The reason we advise not using page aliases is because you end up having dozens of aliases instead of one redirect rule. Less is easier to maintain. Simple as that.

However, GitHub Pages has no redirect facility (and probably never will). So you really don't have any other choice but to use page aliases. This is a key reason why I so strongly advise against using GitHub Pages. It's not a real web host and GitHub chooses to keep it crippled. I prefer Netlify or GitLab Pages, both of which support this feature.

So you really don't have any other choice but to use page aliases.

You could also write an extension that registers page aliases programmatically so you don't have to maintain them in the source.

I think that's good to know! There is no technical drawback of using the GitHub redirects in bulk, but it is indeed a bit of a pain to maintain.

[fhennig]

Update: I have spiked the distributed component. and it works and looks nice! See here: https://deploy-preview-357--stackable-docs.netlify.app/home/nightly/operators/index.html

Using netlify, I have added some redirects so the stable links to content do not break. :heavy_check_mark:

The only problem right now is that the version dropdown doesn't allow selecting old versions of documents anymore, because to antora, the documents didn't exist in older versions (which is technically correct, the documents were in different components). It's difficult to find workarounds for this.

Older versions are now a bit difficult to access.

I've tried page aliases, but antora doesn't pick that up for the version dropdown. The "proper fix" would be to indeed refactor all the old versions, but that is a lot of work.

The options I see are:

• leave it as it is
  • no permalinks are broken, so it's fine
  • older content still accessible with correct links
  • only pain is that you can't easily navigate to it
• backport changes
  • doable for 23.1
  • difficult/painful for 22.11, 22.09, 22.06

[fhennig]

Update: The move to netlify is progressing. We have our current docs mirrored now on https://docs-preview.stackable.tech , which is on netlify.

We decided that it is fine for now to backport changes to 23.1 as that seems easy, but ignore older versions, leave them as they are.

A quick look at matomo showed almost no one accessing the older versions as well.

[fhennig]

Done as far as I'm concerned

look here: https://docs-preview.stackable.tech/

• changes were backported to 23.1
• all links should still work
• old docs are a bit hard do reach but so be it.

If everything is fine, we should switch our proper docs link over to the netlify site.

[fhennig]

Oh, also i fixed all warnings/errors and enabled a build failure on warnings.

[fhennig]

Changes are live now