overhaul documentation completely

[sanmai-NL]

This PR addresses textual as well as technical issues. A summary of the changes is given in the commit message.

The main change is a documentation markup language translation from GitHub Flavored Markdown to Asciidoctor. In my opinion, my translation has delivered promising results. Since the Asciidoctor markup has turned out very powerful, readable and concise, this work/experience could be re-used for future documentation work on other OCaml-related projects, such as the OCaml distribution itself. Some slight issues in which peculiarities of OCaml vs. Asciidoctor syntax have conflicted have been reported upstream.

[sanmai-NL]

The sample include syntax, used at the very end of the manual, does not work on GitHub yet it does in the rendered documents pushed to the repo. Allegedly there is a workaround, but I haven't managed to get that working quickly.

[gasche]

This is very nice work, thanks!

I have a few minor comments on the current PR state:

• Many commits are minor fixes, could you squash them to produce a nicer history?
• I would like to avoid versioning rendered documents in the source repository; on the other hand, it would be useful to have a Makefile to produce the rendered document; eventually we could transform it into a workflow to automatically push the rendered documents to a user-facing website for OCamlbuild. A nice idea with having this explicit rendering step is that we can script it a bit, for example include the git commit hash in the rendered version. Could you remove the rendered documents from the commits, and maybe have a very simple Makefile with build rules? (Eg. one for html, one for pdf). You may want to add some static data meant for rendering (for example a CSS sheet or some HTML preamble) somewhere in the repository, but then I would like to know where this data comes from (who authored it, what the license is, etc.), and it would be best to include an explicit link to the source as a comment in the data files themselves.
• The symbol you chose for note captions does not render correctly with my font (apparently my default font is "Noto Sans"), could you pick something else?. Also, I'm not fond of the death symbol you chose for caution (that's a culturally overloaded symbol with associations we don't control), could you for example just double the warning symbol?

On the rendering side, note that @linding has been independently working on a nice stylesheet for the markdown rendering, see his repository and the rendered document. I'm afraid most of that work would have to be redone after the AsciiDoc switch, but hopefully you can join forces to get a html rendering that you both like.

[sanmai-NL]

• I tried in response to your request, but squashing did not work out well (anymore) and I have limited time to spend fighting git. I think keeping the history beautiful is of limited value given the state of the current history, and that we'd best wipe history out and move it into the ocamlbuild repo at some point.
• I expected your remark about the PDF/versioning, but I chose to present the rendered documents publicly so they can be evaluated in public. After all, much here is a matter of taste. I also initially wanted to mention automatic documentation building, but thought it not immediately relevant. We can look at using a Travis CI for this.
• This tool showed me that Noto Sans and plenty of other fonts do contain glyphs for 'CIRCLED INFORMATION SOURCE' (U+1F6C8): 🛈. GitHub, however, renders some of these Unicode glyphs as colorful emoji, and perhaps something did not work out right for you due to that. The Unicode admonition symbols do not appear in the PDF output though, so I've added texts besides them. The Font Awesome-based icons that Asciidoctor uses normally do not work on GitHub (or in a tty), encouraging me to substitute them with Unicode equivalents in the first place. The texts also helped improve the presentation on GitHub. Please consider that Unicode symbols have semantics more than font icons/images, which could help e.g. blind readers. In a few months/years time, all of them will be displayed as colorful emoji anyway. An alternative could be 'NOTE' (U+1F5C8): 🗈. I do take the potential offensiveness of 'SKULL AND CROSSBONES' (U+2620): ☠ as CAUTION symbol far more lightly than you :wink:, but I replaced it with 'CAUTION SIGN' (U+2621): ☡.

The link you gave to @lindig's styled document actually does not show more styling at all compared to the GitHub-rendered Markdown document, and certainly less than the Asciidoctor document rendered to HTML and PDF. Are you sure this was the work he intended you to reference? @lindig: It would be great if we can keep up the good work together! In fairness, there is also another PR #30 that you did not yet address @gasche. It is possible to restyle the Asciidoctor document using CSS, but I actually think the content now deserves far more priority.

Do you wish to merge the PR? We can address remaining issues later, agree?

[gasche]

Thanks for the careful response. I squashed all the commits into one, and merged the result.

Thanks a lot again for your work. The TODOs that you have added in the text are also very helpful, I will try to correct them over time.

Thanks for the pointer to #30; I did, of course, forget about it. That means that now I have to port it to the new syntax.

[gasche]

P.S.: when doing the merge into the ocamlbuild repository, I intend to keep the commit history. If only, that's more respectful to the contributors whose work is visible through the history.

[gasche]

@sanmai-NL , could you give some advice on the preferred way to reference sections?

It seems that AsciiDoctor generates a default anchor name by mangling the title name, which is good. But if I want to explicitly reference a section, I add a [[anchor]] before its declaration, and then AsciiDoctor replaces the auto-generated anchor name by the one I have given.

This has unfortunate consequences in terms of upward-compatibility of links: if users today refer to a specific section with an auto-generated anchor, and I later decide to point to this section from another part of the document, the auto-generated will become invalid in future versions of the document.

I would suggest to always explicitly give an anchor at each section level (for example reusing the naming scheme I used in the markdown version); this also has the advantage that the anchor is robust to minor changes in the title. Is this standard practice, or is there a better way?

[lindig]

On 19 Apr 2016, at 23:33, Sander Maijers notifications@github.com wrote:

The link you gave to @lindig's styled document actually does not show more styling at all compared the GitHub-rendered Markdown document, and certainly less than the Asciidoctor document rendered to HTML and PDF. Are you sure this was the work he intended you to reference?

The point of rendering of manual.md into a self-contontained single manual.html file was to be independent of the rendering GitHub provides server side. However, the style file indeed mimics the GitHub style.

https://github.com/lindig/manual-ocamlbuild/blob/master/manual.html

— Christian

[sanmai-NL]

@gasche: As an HTML element can only have one id attribute, and the id value is used for cross-references, your observation is not unusual. I think one best never creates long-term references (e.g., URLs shared on mailing lists) to HTML documentation items that have no fixed ID and thus URL. Guarantees about section persistence are hard to give anyway, since also the non-fragment identifier part of the URL, section title, section ID and the mere existence and content of the section can all mutate. For URL references in the context of academic publications, please only share persistent identifiers. In the general case, only link against a specific tag or revision in the repository when linking to any documentation. I would advise that you only assign section IDs to sections that you need to cross-reference, since once chosen a section ID should not mutate (after the project's version has reached >=1.0.0). We may need to rethink the section IDs, the ones I chose are ad hoc.

[gasche]

Thanks for the information. (Note that it shouldn't be to hard to generate dummy <a> elements before the title to create anchors for additional labels.) I think I will decide on a reference scheme that is globally consistent, possibly reusing the one of the Markdown version, and systematically add a reference on all (sub)sections. I cannot guess in advance which section users of the document will want to reference in their URLs -- all of them, I suppose.

[sanmai-NL]

Are you sure about this? I have serious doubts. The section IDs in the HTML version will be consistent as long as the URL resolves to a versioned object, which is easily possible with GitHub as explained. If users host a rendered version of the documentation somewhere else and start sharing long-term references to non-cross-referenced sections, then it is their responsibility to version this document. It already was their responsibility to keep the document available at all. Adding section IDs in advance will make the document source in terms of lines grow linearly in the number of sections. Following your current reasoning, people may also want to link to e.g. a table (this is likely, even). This means the document source would grow linearly in the number of items throughout the document. Are your sure this substantial burden is a good trade-off to allow somewhat careless linking?

Perhaps the real issue is that we still should identify the version of the manual (in a simple form, equal to the version of ocamlbuild) both inside the manual and in its filename. I'm almost done automating the document rendering, a procedure triggered by GitHub releases (tags). To facilitate hosting the rendered documents outside a versioned repository, I will now adapt it to rename the files and add a version postfix.

About the ID naming scheme, please do mind that the namespace of any ID you assign is clear. If I recall correctly, your previous naming scheme did not distinguish e.g. tables from sections.

[gasche]

Adding a label to each object is not an issue, as object declaration already take linear space, so the amortized complexity cost of labels is nil.

We have two choices in how we release rendered versions of the document. We can have the version in the URL, but then (because URLs should be persistent) that means that the storage space grows linearly over time. We can have a single available "last version", and then we need to try to have labels as future-compatible as possible (within the limits of what is reasonable). Also this has the advantage that references self-improve (if I link to a section today, but you read it tomorrow, you'll benefit from fixes/clarifications made in the meantime).

[dbuenzli]

We can have a single available "last version", and then we need to try to have labels as future-compatible as possible (within the limits of what is reasonable).

I think that for the online version this is the simplest and most reasonable thing to do.

While annoying I think it is ok for #id's to sometime fail (though that should be minimized), you can often retrieve what you want by considering the context in which the URI was published. What is more annoying is when the URI itself changes.

[sanmai-NL]

@gasche: That seems a rather theoretical position. A text is a human work of art. Amortized complexity cost is not human measure of growth in a text. The condition of rendered manual URLs being persistent has no implications on storage space, even within a single storage backend (in a practical sense: host). I think you refer to URLs to various versions of the manual staying available requires more storage space than if only a current version is hosted. That is hardly an argument nowadays as even the uncompressed PDF version of manual.adoc is only 777 KiB at the moment. The PR #35 I just submitted renders the manual and adds it to every release, which is proper from the viewpoint of software maintenance where not everyone will be interested in the latest version. Everything is hosted by GitHub here, so you really do not have to worry about hosting rendered manuals.

I shall point out again, that the idea that one can safely and durably link to any section of an evolving document can turn out to be quite misguided. I do see the value of having a URL that points to current version of the manual. But that is simply a matter of configuring a web server, or making a symlink, etc. I'm just contributing to this project here, I am not aware of any plans on your side about self-hosting OCamlbuild materials.