Split documentation into multiple pages

[ubruhin]

Currently the whole documentation is a single page and the TOC contains all subsections of all sections. This makes the documentation a bit confusing (I already removed some subsections from TOC just to improve readability).

I like the concept of ReadTheDocs which has a separate page for every top-level section. And the TOC shows subsections only for the currently opened top-level section.

Unfortunately this is not directly supported by asciidoctor...

[m4c0]

Just for extra context: asciidoctor docs seems to have that by using an extra tool named awestruct.

[m4c0]

I did a little bit of research of current solutions (they are in the order I researched them):

• Jekyll

  • Used by Github Pages
  • Written in Ruby
  • Recommends Liquid for templating (also written in Ruby)
  • Templates are mostly HTML + some YAML
  • Support for Asciidoctor requires a plugin (provided by Asciidoctor)
  • Mostly used for blogs. Blog posts are automatically indexed, but pages aren't

• Awestruct

  • Used by Asciidoctor docs
  • By default, uses HAML as HTML markup. Supports Slim as well. Both are HTML with less tag markup (aka "span" instead of "X")

• Antora

  • Written in NodeJS (recommends 3GB of RAM)
  • Available as a Docker container
  • Supports multiple branches (this enables per-version docs)
  • Supports Asciidoctor out-of-box
  • Antora docs uses Antora only
  • We only need to write one YAML to onboard (remaining bits are Asciidoctor and xrefs)

• Sphinx

  • Written in Python
  • Requires a Python module to integrate with Asciidoctor

• MkDocs

  • Written in Python
  • Maybe there is a Python module to integrate with Asciidoctor

• ReadTheDocs

  • Explicitly does not support Asciidoctor

I am pretty sure there may be other tools out there, but at least that may help starting a discussion.

[m4c0]

I am deeply sorry about the mention spam. Next time, I'll avoid linking references until I have my commits ready for review.

[m4c0]

The research part for Antora is completed. It is a strong candidate (details in the pull-request).

I started to research Jekyll. It sounded like a good candidate as well, since GitHub pages supports it and it is written in Ruby (the same language we currently require for the docs project).

Unfortunately, it is not that simple because its installation process requires a working GCC installation in all platforms. LibrePCB devs most probably have it already, but we can't assume the same for LibrePCB users. This is quite heavyweight to a doc-only contributor.

I am already biased against Jekyll, but I will continue researching it (since I have said GCC environment). Maybe its features are impressive enough to keep Jekyll as a candidate.

[dbrgn]

I am already biased against Jekyll

I'm also biased against Jekyll :smile:

By the way, I Just noticed that hugo (which we use for the current website) supports asciidoctor as well: https://gohugo.io/content-management/formats/ The advantage of that would be that Windows users could download a single static binary and it would just work. Might also be an option worth checking out.

[m4c0]

I'm also biased against Jekyll 😄

I am giving up on Jekyll... As a static builder for a blog platform, it is amazing. For technical documents, it lacks good out-of-box templates. I found out docs.twitch.tv and aws-amplify.github.io uses it. Only Amplify has it as open-source and the source is scary: too much templating and such. It will take a lot of effort to reach any minimal result.

By the way, I Just noticed that hugo (which we use for the current website) supports asciidoctor as well: https://gohugo.io/content-management/formats/ The advantage of that would be that Windows users could download a single static binary and it would just work. Might also be an option worth checking out.

Well, I am running out of options, therefore, I will check it out now... :)

[m4c0]

For completeness, there is an interesting player in this league: Gollum. It is a "wiki over GIT". I may check it out later.

EDIT: it is written in Ruby (which does not introduce new dependencies) and supports versioning.

[m4c0]

@ubruhin - As I investigate Hugo further, I went down a little deeper in Asciidoctor's own docs. It seems Asciidoctor is much more extensible than I expected. Now, after reading more about its capabilities, I do have some ideas with Asciidoctor extensibility that I want to investigate further.

Before diving into those, I would like to step back and confirm what are the pain points we are trying to solve. From my understanding:

1. TOC is too big. This is the same issue in Asciidoctor User Manual. If they had a collapsible tree per sectlevelX, then sections like "Extensions" and "Build Integrations and Implementations" would be more exposed. Same for us, specially when our docs grow.
2. Offline version (preferably PDF). Because it is useful for our users. As long as it is easy to download and browse offline, PDF is not mandatory.
3. Per-section HTML. Eventually, the output HTML will be too large due to images. Question: if we assume <img> tags with specified height and a really good TOC navigation, is this still a problem?
4. Hot-links. That little paragraph icon with a direct link to the referred section.
5. Simple to contribute. We want to avoid asking contributors to install complex stacks.
6. Versioning. This was a nice idea from Antora: an archive of manuals per LibrePCB version.
7. Language support. Because eventually we need to think about translating our docs to different languages.

Am I missing something?

[ubruhin]

Finally I got some time to think again about this issue :slightly_smiling_face: Somehow I still really like the output of Antora. But as discussed, getting Antora running on Windows is a pain. I see two options to work around this:

1. Use Antora for the hosted documentation, but allow building docs locally with only Asciidoctor. In the simplest case, it's a matter of adding an additional index.adoc used only for this purpose - but I'm not sure if this really works well or if we would get issues (like broken hyperlinks, wrong image paths etc.) if the source .adoc files get processed with two different tools - depends on whether the .adoc files are fully tool-independent.
2. Maybe we could even bundle the required tools by ourselves, e.g. creating a ZIP file which contains Antora with all its dependencies? Not sure if that's possible and legal...

But on the other hand, the Hugo output is also not bad and much simpler to get it set up on Windows :see_no_evil:

@ubruhin - As I investigate Hugo further, I went down a little deeper in Asciidoctor's own docs. It seems Asciidoctor is much more extensible than I expected. Now, after reading more about its capabilities, I do have some ideas with Asciidoctor extensibility that I want to investigate further.

What do you mean exactly, writing a custom Asciidoctor extension?

2. Offline version (preferably PDF). Because it is useful for our users. As long as it is easy to download and browse offline, PDF is not mandatory.

Actually I think the offline documentation does not even need to contain everything the hosted documentation contains. Probably it would make more sense to provide only the actual user manual, but omit things like these:

• Library conventions are not related to the tool, they are only related for our official library repositories. So I'd host them only online, but not in a offline user manual.
• Same applies to library contributing instructions, once we have documentation for them.
• Installation instructions might also not be needed in a offline user manual. For example if we bundle the user manual with the application some day, it doesn't make sense to include installation instructions since then you already installed LibrePCB.

3. Per-section HTML. Eventually, the output HTML will be too large due to images. Question: if we assume tags with specified height and a really good TOC navigation, is this still a problem?

I wouldn't call it a real problem, but for me, many small pages somehow feel much more comfortable than a huge single page...

Am I missing something?

I guess no, very good summary of requirements! :+1: