Style the html manual, changing type and layout

[charlesetc]

Hi!

So this commit makes major visual changes to the OCaml manual. I'm very happy to take any feedback and make changes.

I've got a preview up of these changes. This is compared to the current version.

The header font is Fira Sans. I checked it into this repository, so the font would be served from OCaml's servers. There is an argument to be made for using Google Fonts instead, since it can make things faster through caching. I'd be interested in hearing what people think about that.

The only "hack" here is the following snippet:

\newstyle{body >
          div:first-child >
          span:first-child >
          span:first-child}{
    font-family: "Fira Sans", sans-serif;
}

Which selects the "The OCaml System Release 4.08 ..." heading to make it a different font. Unfortunately there's no id or class by which to select this — if anyone knows how to add one that'd be awesome!

In reference to the recent change (https://github.com/ocaml/ocaml/commit/49abafc5872f6be347cdb72a8a4ecccd229b6118) of the manual, this keeps the max-width for the body, but adds a margin: auto so that the body is centered. I think this looks slightly better and also more like the current manual. On small screens, when max-width doesn't apply, this also adds a percentage padding. You can go to preview_docs.narwhal.space on a phone to see how it looks.

Anyways, I'm glad to make any changes people see fit :cactus:

[xavierleroy]

I'm glad to see efforts to improve the look and legibility of the online manual. However something is very wrong with the OCaml code snippets, e.g. the first example of section 1.1, 1+2*3. The original rendering has colors and a # prompt to distinguish input from output. The new style has none of that and is unreadable.

[charlesetc]

You're right! I'll see what I can do about that.

[charlesetc]

It's fixed now! I had the preview showing outdated code

[Drup]

I really like it, but as someone with terrible eyesight, I would prefer if the background was not pure white. While it looks good and modern, pure white background is very tiring for the eyes. I also tend to like when code blocks have specific background.
For example, RWO uses a slightly beige background and darker code blocks, which is far less aggressive.

Similarly, you might want to tone down the electric blue a bit.

Misc remarks:

• Some definition overflow, such as http://preview_docs.narwhal.space/extn.html#sec236
• I believe the links to modules in the documentation are not rendered correctly. See Int32 in http://preview_docs.narwhal.space/core.html#sec549
• The small yellow arrows certainly have a vintage charm, but do not really correspond to the style anymore. :)

The jump to the library documentation is very jarring now. I suppose that is your next step?

[bluddy]

That's some good css hacking. Adding classes isn't possible without modifying hevea, which isn't on github. I'm really impressed by what you've been able to accomplish just using this method.

[pmetzger]

I think that using Google's free Fonts CDN for Fira Sans will be better than serving it off of our own servers — it will load faster.

[pmetzger]

Unfortunately there's no id or class by which to select this — if anyone knows how to add one that'd be awesome!

So I've also been working on doing something similar, and Lambda Soup can be used to postprocess the manual quite nicely for this purpose. One of the changes I've made in my own hacking has been to replace the arrows at the tops and bottoms of the pages to include SVG arrows in their place. You can also nicely alter the CSS classes in the way you wish. Code to do this is quite compact and easy.

[pmetzger]

Oh, and in my own experiments I've used Fira Sans for the headers just as you did, and EB Garamond for the body text. EB Garamond is one of the nicest serif fonts available on Google Fonts. Try it, it's quite nice.

[pmetzger]

On the background, I find the beige a little too tinted. Perhaps F7F7F7 instead?

[pmetzger]

The small yellow arrows certainly have a vintage charm, but do not really correspond to the style anymore.

So I have two options to suggest there, both easily added by the use of Lambda Soup to edit the HTML.

1. Just say Back, Up, Forward.
2. Use Google's "Material Design" SVG arrow icons, which are very nice and scale perfectly to every resolution since they're vector art:

https://github.com/google/material-design-icons/blob/master/navigation/svg/design/ic_arrow_back_24px.svg

https://github.com/google/material-design-icons/blob/master/navigation/svg/design/ic_arrow_upward_24px.svg

https://github.com/google/material-design-icons/blob/master/navigation/svg/design/ic_arrow_forward_24px.svg

[pmetzger]

On style.css for stdlib, I was working on this yesterday and struggling to get everything quite right because the CSS classes are so eccentric, but it should be possible to get something quite nice and consistent with the rest of the manual by using a touch of Lambda Soup to postprocess and change the class structure.

[pmetzger]

BTW, @aantron was helping me with Lambda Soup assistance, so I'm tagging him here in case there are Lambda Soup questions.

[charlesetc]

Haha thanks @bluddy! @Drup thanks for the great suggestions! I've added a light grey background (@pmetzger F7F7F7, thanks!) to the body, and a beige background + light outline to code blocks. This code block change is another big visual change, so we should be sure about it. I also changed that light blue on visited links to a darker blue which should be better. I'm not sure what's wrong with the links to modules like Int32 — it looks to my like it's the same in the current version. Am I missing something?

Unfortunately there's already a tiny bit of definition overflow in the current manual. Bumping up the font size, and limiting the body adds to that. It's also arranged with tables so I haven't yet found a way to get them to wrap around or scroll... but I'll keep trying.

Also good point about the standard library formatting! I've just added styling that does it. See this Str module.

I'm actually kind of fond of the arrows, but they are obviously dated. I don't know anything about Lambda Soup but it does seem like it would take some kind of post-processing to do it. @pmetzger everything you're saying about lambda soup sounds awesome and is definitely what this needs!

I just tried EB Garamond and it's definitely a nice font but looks a little strange here so I left it out for now

[charlesetc]

Oh also I switched to the Google CDN :)

[pmetzger]

The reason it looked strange in the libref stuff is because of the horrid class problems in the generated ocamldoc. Try using EB Garamond only in the hevea generated manual for now and you'll be happier. We'll need Lambda Soup to clean up the ocamldoc classes in order to do real CSS font styling there.

[pmetzger]

So Lambda Soup may be found at: https://github.com/aantron/lambda-soup

You will note that the documentation for Lambda Soup is itself postprocessed with a Lambda Soup program that is given in the github repository.

[pmetzger]

(I'll be back tomorrow. Big kudos on doing this work!)

[charlesetc]

Wow Lambda soup looks awesome and it's ocamldoc site is beautiful!

Thanks :smile: I'll also be back tomorrow to:

• Try to fix the overflowing BNF's
• Add EB Garamond (having trouble atm)

[gasche]

For the record, I'm not completely excited at the idea of post-processing generated documentation. Using Lambda Soup is fine for experiments, but if we decide that we like a certain style, why not change the code generation instead to get it from the scratch? Post-processing would make the manual/documentation build harder to maintain.

[Octachron]

@charlesetc :

Which selects the "The OCaml System Release 4.08 ..." heading to make it a different font. Unfortunately there's no id or class by which to select this — if anyone knows how to add one that'd be awesome!

Sorry, I missed this point on first reading. One could fix this problem by making the latex code more semantic and replacing the \begin{center}...\end{center} used in the main title by a new maintitle environment and then adding the right definition to macros.tex and macros.hva: example

[Drup]

@charlesetc Thanks, this is much nicer!

I agree with @gasche, in particular regarding the API documentation. We can tweak the output of ocamldoc whichever way we want, might as well avoid doing postprocessing. Also, improvements in ocamldoc styling can easily benefit odoc as well.

Notably, Having a side panel with an index for the manual would be really cool.

[pmetzger]

For the record, I'm not completely excited at the idea of post-processing generated documentation. Using Lambda Soup is fine for experiments, but if we decide that we like a certain style, why not change the code generation instead to get it from the scratch?

That requires a lot more learning to do. Fixing the HTML is easy. Fixing ocamldoc requires learning how the inside of ocamldoc works and that's much less obvious. I would recommend we not postpone fixing up the appearance until that work gets done. Instead, I would suggest we do this in two steps:

1. We do some postprocessing now, just to figure out what we want and get things looking nice for the short term. That gets committed for now, because it will make the documentation more readable for people now. The nice thing about code is it isn't stone, we can get rid of it at will in a month or six months.
2. A volunteer who understands ocamldoc and hevea fixes them up to generate better things, and then the postprocessor is thrown out, but they don't have to rush to do this because we have the postprocessor as a crutch for the moment.

[pmetzger]

Notably, Having a side panel with an index for the manual would be really cool.

We can do that pretty quickly with Lambda Soup, and we can fix it "correctly" over time. @aantron has an example in his own documentation for Lambda Soup.

[Chris00]

We do some postprocessing now, just to figure out what we want and get things looking nice for the short term.

For the library documentation (as opposed to the manual itself), if there are hesitations to merge post-processing in the repo, an opam package with a small tool would be nice (and useful for other people as well).

[gasche]

I would suggest we do this in two steps:

1. We do some postprocessing now, just to figure out what we want and get things looking nice for the short term. That gets committed for now, because it will make the documentation more readable for people now. The nice thing about code is it isn't stone, we can get rid of it at will in a month or six months.
2. A volunteer who understands ocamldoc and hevea fixes them up to generate better things, and then the postprocessor is thrown out, but they don't have to rush to do this because we have the postprocessor as a crutch for the moment.

What will predictably happen in that scenario is that no one will ever do step (2), and we will end up with a build system that is even more baroque than it is today. We will still be there maintaining the manual build ten years from now, and I think it really is in our collective interest for me to look a bit more annoying right now, and try to get someone (not necessarily the PR author) motivated enough to make the rendering fixable at the source now and not in six renewable months.

[Octachron]

Another point is that is not like hevea or ocamldoc are black magic that noone understands. Using documentation post-processing in the explorationphase sounds like a good idea. But once there is a clear target, I would also argue that the changes should be backported to hevea or ocamldoc. For instance, I imagine that it would benefit to all hevea users to have a more modern choice of navigation arrows.

[nojb]

What will predictably happen in that scenario is that no one will ever do step (2), and we will end up with a build system that is even more baroque than it is today. We will still be there maintaining the manual build ten years from now, and I think it really is in our collective interest for me to look a bit more annoying right now, and try to get someone (not necessarily the PR author) motivated enough to make the rendering fixable at the source now and not in six renewable months.

Seconded. The manual is hard enough to build under Windows, no need to make it impossible to do so.

If someone could explain what are the problems/shortcomings with the current ocamldoc/hevea output, people who are familiar with them could look into what is required to do the necessary modifications.

[gasche]

@charlesetc: could you add a Changes entry, in the 4.07 part, section "Manual and documentation", crediting people that contributed to the patch?

Also, it would be nice to integrate @Octachron's suggestion in https://github.com/ocaml/ocaml/pull/1757#issuecomment-386525751 on how to get a proper id/class for the main title.

(My gut feeling is that the PR could be merged after this. In particular, I'm at peace with the way the extra <meta> tag is handled.)

[Octachron]

In particular, I'm at peace with the way the extra tag is handled.

( in fact, this fragment is directly extracted from the hevea manual on how on to add meta tags to the <head> tag )

[pmetzger]

What will predictably happen in that scenario is that no one will ever do step (2), and we will end up with a build system that is even more baroque than it is today.

My fear is if we don't do it in steps, it won't happen, and we'll end up with the ugly manual forever!

How about we compromise by finding an expert on the ocamldoc code who is willing to do the work with @charlesetc (and possibly me) so that we get the best of both?

[pmetzger]

BTW, if we want to change the arrows to SVG in hevea itself, how do we do that?

[pmetzger]

@nojb:

If someone could explain what are the problems/shortcomings with the current ocamldoc/hevea output, people who are familiar with them could look into what is required to do the necessary modifications.

The hevea output is almost okay. There's some things, like the use of tables for centering headings in the table of contents, which really should go — if one wants such things, one does them in CSS — and there are some other infelicities like the arrows being output. The biggest issue, which is actually possibly hard in hevea, is generating an index/ToC sidebar, which would be easy in postprocessing.

The ocamldoc output is very messy. It uses tables a bunch, but worse, it has a really baroque set of CSS classes, and it's hard to set things like alternative fonts or stylings well because of this.

[pmetzger]

BTW, let me reiterate, as strongly as @gasche fears that if we do postprocessing it will become permanent, I fear that if we don't do it, we won't get a pretty manual any time soon. The way to fix this so everyone is happy is to get people who know how to fix the ocamldoc code and who know hevea to help while we have momentum on the project.

[gasche]

Let's take a step back out of this conversation on post-processing. There is no processing in this PR and nothing to change in either hevea or ocamldoc to get the desired output.

If/when we get a PR that does require processing, we can see how things work out. Some points that might reassure you:

• @Octachron seems quite willing to give feedback and advice and is sort of our ocamldoc expert (but then, he wasn't three years ago, and never volunteered for the job, anyone can do it with OCaml knowledge and a bit of curiosity in getting problems solved).
• I just sent an email to the Hevea author, @maranget, to ask him whether we could have a git repository for Hevea where to send prospective pull requests.

[Octachron]

It uses tables a bunch, but worse, it has a really baroque set of CSS classes, and it's hard to set things like alternative fonts or stylings well because of this.

I might be responsible for this baroque set of CSS classes, which also means that I could also remove/reorganize it.

[pmetzger]

There is no processing in this PR and nothing to change in either hevea or ocamldoc to get the desired output.

Well, sort of. As you can learn from the thread above, @charlesetc can improve the CSS for the main parts of the manual most of the way without postprocessing. However, to fix the libref portions, we need to fix the ocamldoc generated classes. Otherwise it turns out to be quite hard to clean up the CSS.

However, in the spirit of "one step at a time", I'd be happy if we fix the main parts of the manual with just the small CSS changes in question and left fixing the rest for another PR.

[bluddy]

Another potential question to consider is, should hevea be used at all. It's far from clear to me that the very heavy syntax of latex is the right choice for a user manual. Most systems (such as Sphinx) use a far lighter format, such as RestructuredText, and have output in latex and html. The ocamldoc format itself should be capable of recreating everything in the manual, and that would reduce us to just one tool.

[pmetzger]

I think a migration to Sphinx would be an excellent idea. The Coq project moved to Sphinx and it has made their lives a lot easier. However, I'm a big believer in doing things incrementally, and we can make the manual look much better with very little work right now. I'd suggest we do the small changes for that, and then consider arguing over a move to Sphinx later. (For the record, if an argument to the effect that we should move to Sphinx comes up, I'll likely be on the pro-Sphinx side. But I think we need to eat the elephant in small pieces.)

[dra27]

I completely agree with other devs on not post-processing here.

@pmetzger: - ocamldoc may be a fearfully large looking codebase, but grep is your friend. A few second's search reveals that ocamldoc/odoc_html.ml is, not too surprisingly, where all this is generated - that file consists of relatively straightforward-looking printf-like statements. IIRC the manual compiles using the ocamldoc contained in the build tree, so it looks fairly easy to me to work on changes to the HTML generator. For a more advanced solution, you could try adding a functor in ocamldoc/odoc_gen.ml to have some kind of NewHtml backend, in order to maintain compatibility, but that's not necessary to start with.

The biggest issue, which is actually possibly hard in hevea, is generating an index/ToC sidebar, which would be easy in postprocessing.

Pausing slightly at the observation of "actually possibly", and with the caveat that I haven't actually looked at its code, it is very surprising to me that a program which must process a LaTeX table of contents in order to convert it HTML could have any difficulty doing this...

[pmetzger]

Pausing slightly at the observation of "actually possibly", and with the caveat that I haven't actually looked at its code, it is very surprising to me that a program which must process a LaTeX table of contents in order to convert it HTML could have any difficulty doing this...

I'm sure it is possible. It's just that I don't see immediately how to do it without changing code rather than configuration. That said, I may be quite wrong.

[pmetzger]

By the way, I've put a quick-and-dirty Lambda Soup hack to postprocess the arrows into nicer SVG here: https://gist.github.com/pmetzger/4eaf8403a0e2cd3f6176d9f7367c8176

Results of postprocessing the ocamldoc.html file can be downloaded from here: https://gist.githubusercontent.com/pmetzger/bf31530b572d2a071efd459004874dee/raw/9ebab1eda3296b87ba68ae8bd92d84a24cbbe5b8/ocamldoc.html

If hevea can be made to generate the SVG arrows directly, then this sort of thing will not be necessary.

[pmetzger]

Also, Steffen Smolka, in this thread, came up with a nice grey background color scheme for source code examples in the main (not libref) part of the manual. (Note that other proposed schemes are also in that thread, which was started by @Octachron, who came up with some interesting ones of his own.)

@import url('https://fonts.googleapis.com/css?family=Source+Code+Pro');

div.toplevel {
  padding: 1ex 1em;
  background-color: #EEE;
  font-family: 'Source Code Pro', monospace !important;
}

div.caml-input {
  color: #11A;
}

div.caml-input::before {
  content: "#";
  color: black;
}

div.caml-output.ok {
  color: green;
}

div.caml-output.error {
  color: red;
}

[pmetzger]

By the way, enough people in discussions on IRC and discourse seem to have concerns about availability of fonts offline that it is probably going to be necessary to bundle/serve the fonts, rather than to use the Google Font CDN for it.

[charlesetc]

So I've included @Octachron's suggestion to add a .maintitle class and I've also added classes to the tables that were overflowing (in the current version too), so now they have a horizontal scroll bar. I also went back to hosting fonts on the ocaml servers instead of google's, and have better font support in general.

I think the light grey for text boxes looks good on a white background, but I like the beige that's in the preview currently, since the body also has a light grey background. What do people think?

With respect to future plans, I totally agree that moving to Sphynx or another documentation generator would be great. It also might be nice to be able to release the manual changes more rapidly than ocaml compiler changes? Maybe a "current" manual or something, I'm not sure.

[gasche]

I merged this PR, and also cherry-picked in 4.07 ( 833abd351925612a2c000a927e8852df763a7fdf ).

Thanks for the nice work!

[gasche]

It also might be nice to be able to release the manual changes more rapidly than ocaml compiler changes? Maybe a "current" manual or something, I'm not sure.

We have discussed this before. In principle there is an agreement that this is a good idea, but in practice we have never really felt the need to do special documentation updates. With a six-month release cycle, the manual is already updated somewhat frequently (for example, 4.07 is supposed to be released this month or shortly after). More frequent updates cannot be completely automated (we don't want an update the last-release manual to include documentation of library functions or language features that are only available in the development version).

In summary, please feel free to request an update to the manual/documentation whenever you feel this is justified by a big enough change.

[dbuenzli]

@gasche I understand your desire to hastily merge this to avoid further bikeshedding. However I would like to suggest to @charlesetc that a 19px for the body text is way too large.

Please have a look at many websites where you read a lot, (e.g. wikipedia, github) the font size for the body text is never that large. The information density one gets on a laptop on a screenful with the the proposed font size is nearing zero.

Why not stick to the browser default of 16px or, even better, respect the browser's user preference and use 1em and a relative design ?

[pmetzger]

19px is not too large I think — I sometimes argue for larger — but it is true that it's useful to rig things up to scale depending on the size of the browser. There's an excellent example of this in the CSS for Butterick's book Practical Typography, with the point size and width being adjusted depending on the pixel sizes.

All of this is detail of course, and can be improved with time.

[pmetzger]

BTW, is there a version of the current style up for viewing or do I have to build the manual from source to look at the latest version?

[pmetzger]

Assuming for the moment that the version that's up is the preview, there are still a bunch of nits we probably need to fix. For example, the table that put out "type spec" in this: http://preview_docs.narwhal.space/libref/Arg.html is pretty eccentric, with the comments getting far too much width while the actual types get almost no space.

We might also want to bring in EB Garamond as a specified body font for the main part of the manual rather than leaving it to whatever the browser happens to have around as a default.