Documentation problem: including build artifacts in the repo

[lukego]

There is a problem with our current method of documentation generation: that we checkin documentation build artifacts to the repository rather than only source code. This is messy. If you want to change an ascii-art diagram in a README file then we expect you to install Java (!) so that you can regenerate the PNG file using Ditaa. I think it would be much better to include only source in the repository.

How about if we would create a separate repository for the documentation, e.g. snabbdocs, and store the documentation there? This could in principle by managed by SnabbBot: it could build the documentation for PRs/merges/releases and check that into the snabbdocs repository. Then we would simplify the build dependencies for developers and always have the documentation available in various formats (e.g. markdown, html, pdf, epub).

Hydra (Nix) may also be an easy (easier?) way to build the artifacts than extending SnabbBot but with the difference (downside?) of storing the results on a filesystem of some machine rather than in a Github repo.

cc @eugeneia @wingo @kbara @domenkozar

[domenkozar]

Separating docs from source has (a possible) big downside, separating source changes from documentation changes. It's harder to pinpoint the two. It's better to not commit build artifacts in the first place, since that makes the repository quite big.

I'd recommend using a simple Markdown builder like http://www.mkdocs.org/ and https://readthedocs.org/ (supports multiple versions and multiple formats) for hosting the documentation, they're doing an awesome job keeping that site running. We could hook readthedocs into git so it would build the docs, but if that won't be possible due to dependencies we can use Nix+Hydra to publish them. Very SaaSy, but I use this workflow for all my projects and so far it worked well.

[eugeneia]

I think that the current documentation handling is extremely simple, sophisticated and scalable. I disagree with the implication that having the artifacts in the repository or installing ditaa is a problem. I am intuitively against all suggestions in this thread so far because I expect none are thought through to the end and I expect them to complicate things and/or deprive us of valuable working features.

• SaaS please no. I tried to make mkdocs and readthedocs consume Snabb Switch's markdown files and ended up trying to decipher opaque error messages from their Python/Ruby/Whatever web apps. I would have considered them an option if they managed to build something useful form the Markdown in the Snabb repository without intervention, but even that would probably have been a mistake. I will not email a SaaS support hotline, so I am against SaaS.
• Split repositories: I expect this would be equivalent to just throwing all the docs away and letting them rot. I don't see the upsides to this administrative overhead, I don't see how we “simplify the build” by doing that. You only need ditaa (which is trivially easy to install on any OS, including Slackware) if you want to change a diagram

I propose to cover https://github.com/SnabbCo/snabbswitch/pull/754#issuecomment-183353494 in this discussion as well:

I also adopted an experimental documentation style here: to write real documentation in markdown syntax but include it inline at the top of the source file. The Makefile could extract this for inclusion in a manual. I still think the key to high-quality documentation is to put it somewhere that people will see it every day. Flame on! :-)

I don't think source file headers are more prominent than README.md files. I don't think people should be tricked into seeing the docs. If someone has the desire to read the docs I am sure they will find them. I don't think the key to high-quality documentation is to move it into source files. I think extracting documentation from source code is a waste of time and introducing pointless complexity.

[lukego]

@eugeneia Fine: you reject all these ideas as invalid.

However: the documentation does need more enthusiastic users and contributors. How do we make that happen?

[lukego]

Can also be that I simply don't understand how documentation is supposed to be consumed. Is there a canonical recommended way to consume the documentation?

I know that I can read markdown sources in Emacs, or that I can browse online via Github, or that I can manually build a HTML file and copy that somewhere my browser can access it. Which of these am I expected to do? (Or should I be doing something else?)

[kbara]

I'd be tempted to say "whichever you prefer of those options", assuming they all work.

[lukego]

@kbara None of them work to my satisfaction and I would like to know which is the most important to fix.

[kbara]

Fair. My preference is for the first of those, but that may well be a minority view.

[lukego]

(Historical note: Snabb Switch was originally a kind of literate program that formatted comments in source files as markdown and interspersed them with code.)

[eugeneia]

@lukego The reason I did not propose alternatives is because I don't really get what this issue is about. It doesn't spell out a set of requirements I want to try to satisfy, and I am not convinced there is actually a bug here. I think this issue is a junction of the numerous documentation related complaints over the past weeks. To effectively act on the complaints we need to work out requirements and separate issue topics. I see two mostly unrelated topics in this thread:

• How we write documentation: I am fine with the status-quo, but given a new set of requirements I am willing to find a satisfying fix. However I have to say, I don't find the implicit requirements @lukego stated in this issue particularly compelling (“Do not require Java to render diagrams”, “Do not store artifacts in repository”). I see many benefits that depend on our use of ditaa and having the docs where they are now, and only cosmetic benefit to the stated requirements. To me that sounds like much work for little effect.
• How we read documentation: I am more inclined to focus on this topic and set requirements ignoring “How we write...” because this topic should dictate most of its requirements. I still think we should provide canonical URLs to pre-rendered HTML/PDF/EPUB versions of the docs.

I can do the above on davos using a simple cronjob or hacking it into SnabbBot (I am somewhat inclined to separate CI and doc rendering but @lukego pointed out that having a link to the updated docs for PRs would be a nice thing). I am thinking of a new bot “SnabbDoc” that runs in parallel to SnabbBot and pushes links to HTML docs to GitHub PRs.

[lukego]

@eugeneia How about this for a more concrete plan:

1. Define what is the canonical manual for Snabb. This could be a local directory in the source tree, a Github URL, a HTML file, a PDF file, etc but we should decide which of those things is primary.
2. Define how to access it e.g. a canonical URL for finding a relevant and well-defined version.
3. Treat any defects in this format of the manual as bugs and fix them.

This would make me happy: I could identify the manual, I could refer to it easily, and I could contribute improvements to it.

So question for you @eugeneia: does this sound like a reasonable plan and do you see a way forward to lead us down?

[eugeneia]

@lukego Sounds perfect! I think the primary should be a standalone HTML file at doc.snabb.co/<ref|branch|tag> or similar. Packagers can include the documentation by using make docs.

I would make this happen with:

• doc.snabb.co pointing to davos:80
• Hack up a frugal little brother of SnabbBot that does all the documentation rendering, I think this will yield much effect with little code

[lukego]

Great! I can totally get on board with having a stand-alone HTML manual as the primary documentation that we give loving attention to and prioritize above all other mediums. I think you are right in your comment https://github.com/SnabbCo/snabbswitch/pull/764#issuecomment-187191216 that we can still keep the markdown sources Github-friendly anyway.

How about Github Pages for hosting? I am not in favor of using a lab server in my basement as a production webserver for the project. I also see davos as long overdue for migration to NixOS... but that is a separate topic :-)

[eugeneia]

Fixed in #805.