[Suggestion/Proof of Concept] Documentation upgrade

[marcriera]

I have been thinking lately about openBVE aspects that could be improved, and the official documentation is one of them. Being official, it is the best source of information when developing for openBVE and is checked by everyone, but it has several issues:

1. It is written in plain HTML: I cannot imagine how tedious it is to update the documentation each time a new feature is added (which has been happening more frequently lately). The files are full of HTML tags and embedded styles, and to create a consistent look across pages it is necessary to copy and paste a lot of code.

2. The design could be improved: this is related to the first point. Since the creation of the documentation, the design has been the same, and it has begun to feel out of date. Improving it is currently possible, but would involve modifying a huge amount of HTML tags and code, so it is not worth it.

3. It is not available in languages other than English: while it should be possible to translate the current HTML files easily with a CAT tool, the files are anything but attractive to translate. Being myself a content developer, I get asked quite frequently about documentation in languages other than English, and I am unable to provide them with any other good solution than using machine translation.

After some digging and testing, I think Hugo would be of great help here. It is a static site generator that can be highly customised and is super fast. I have some experience with it and I have successfully created three sites with it, with the most complex of them being dedicated to openBVE content (see here).

For some days I have been working on a proof of concept of how a Hugo-based upgraded documentation would be. You can check the source code here and a live deploy here. I have only fully converted the "B3D object" and "Vs. Trainsim" pages (the rest are placeholders), but the first one contains most of the design complexities found in the documentation and serves as a good example.

This demo showcases the following:

1. Simple and human-readable files: content files are written using Markdown. HTML tags can be used directly in it too, but should not be necessary for most things. The Markdown files also include a front matter defining some page settings, such as the title, the menu position (weight) or whether it should include a table of contents.

2. Design templates: the design is separate from the content and stored in templates, which are common for all pages. These templates are HTML code with Go code, so it is possible to generate HTML parts on the fly, such as menus. CSS styling is also used. The separation from the page content allows to alter the design of the documentation at any time without needing to alter the content.

3. Shortcodes: sometimes it is necessary to format part of a page's content in a specific way, such as openBVE comments, which currently are printed in red text inside grey boxes. Hugo shortcodes make it possible to store the HTML tags separately and call them using a special simple tag, making the content files cleaner. This is used in the demo for commands and notices (yellow and red boxes).

4. Multilanguage support: the content is the same for both languages, but the demo has a language selector that allows the user to switch between English and Catalan. Content files are stored in a separate folder according to the language ("en" and "ca"), and Hugo generates two target folders with the content for each language; the language switcher links them.

Hugo handles all of this and generates a ready-for-use folder with the HTML files in less than a second by just calling it, everything is set up in the configuration file (which is not complicated at all). Furthermore, internal links (including the language switcher) are relative, so it is still possible to have the documentation both in the website and bundled with the program for offline use. The only additional step required is the generation of the target files.

Should you consider this a good idea, I will take care myself of converting the rest of the documentation, setting everything up and maintaining it. The main program source code is too complicated for my basic programming skills to contribute to it, but I would be more than happy to help with the documentation.

[leezer3]

That's very interesting.

Looks OK on the desktop, but the menu eats the whole screen width on the phone.

Let me take a proper look over the next day or two....

On Wed, 15 Aug 2018, 19:29 Marc Riera, notifications@github.com wrote:

I have been thinking lately about openBVE aspects that could be improved, and the official documentation is one of them. Being official, it is the best source of information when developing for openBVE and is checked by everyone, but it has several issues:

1.

It is written in plain HTML: I cannot imagine how tedious it is to update the documentation each time a new feature is added (which has been happening more frequently lately). The files are full of HTML tags and embedded styles, and to create a consistent look across pages it is necessary to copy and paste a lot of code. 2.

The design could be improved: this is related to the first point. Since the creation of the documentation, the design has been the same, and it has begun to feel out of date. Improving it is currently possible, but would involve modifying a huge amount of HTML tags and code, so it is not worth it. 3.

It is not available in languages other than English: while it should be possible to translate the current HTML files easily with a CAT tool, the files are anything but attractive to translate. Being myself a content developer, I get asked quite frequently about documentation in languages other than English, and I am unable to provide them with any other good solution than using machine translation.

After some digging and testing, I think Hugo https://gohugo.io/ would be of great help here. It is a static site generator that can be highly customised and is super fast. I have some experience with it and I have successfully created three sites with it, with the most complex of them being dedicated to openBVE content (see here https://en.bvebarcelona.cat/).

For some days I have been working on a proof of concept of how a Hugo-based upgraded documentation would be. You can check the source code here https://github.com/MarcRiera/openbve-documentation and a live deploy here https://openbve-documentation.netlify.com/en/. I have only fully converted the "B3D object" and "Vs. Trainsim" pages (the rest are placeholders), but the first one contains most of the design complexities found in the documentation and serves as a good example.

This demo showcases the following:

1.

Simple and human-readable files: content files are written using Markdown. HTML tags can be used directly in it too, but should not be necessary for most things. The Markdown files also include a front matter defining some page settings, such as the title, the menu position (weight) or whether it should include a table of contents. 2.

Design templates: the design is separate from the content and stored in templates, which are common for all pages. These templates are HTML code with Go code, so it is possible to generate HTML parts on the fly, such as menus. CSS styling is also used. The separation from the page content allows to alter the design of the documentation at any time without needing to alter the content. 3.

Shortcodes: sometimes it is necessary to format part of a page's content in a specific way, such as openBVE comments, which currently are printed in red text inside grey boxes. Hugo shortcodes make it possible to store the HTML tags separately and call them using a special simple tag, making the content files cleaner. This is used in the demo for commands and notices (yellow and red boxes). 4.

Multilanguage support: the content is the same for both languages, but the demo has a language selector that allows the user to switch between English and Catalan. Content files are stored in a separate folder according to the language ("en" and "ca"), and Hugo generates two target folders with the content for each language; the language switcher links them.

Hugo handles all of this and generates a ready-for-use folder with the HTML files in less than a second by just calling it, everything is set up in the configuration file (which is not complicated at all). Furthermore, internal links (including the language switcher) are relative, so it is still possible to have the documentation both in the website and bundled with the program for offline use. The only additional step required is the generation of the target files.

Should you consider this a good idea, I will take care myself of converting the rest of the documentation, setting everything up and maintaining it. The main program source code is too complicated for my basic programming skills to contribute to it, but I would be more than happy to help with the documentation.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/leezer3/OpenBVE/issues/260, or mute the thread https://github.com/notifications/unsubscribe-auth/AHVQNTwQ6pYhhGLDybaNCsKch3l8kt9uks5uRGijgaJpZM4V-lbX .

[marcriera]

The design is very basic, I made it look like the original but it is not finished at all, just enough for the preview (the current documentation is not really mobile friendly, either). The theme can be easily altered at any time and that includes adding mobile-specific features such as a burger menu. It is definitely possible.

[leezer3]

So, I'm in two minds over this.

For the most part, the raw data is more readable, which is a good thing.

However, in order to get the formatting correct, it's using either Hugo specific tags or raw HTML. IMHO the point of markdown is that it's (or at least should be) totally user-readable from a generic console based text-editor.

As a point for the future also (this and for that matter the main documentation in whatever form), we need to consider a way to support the HTML5 Translate attribute: https://www.w3.org/International/questions/qa-translate-flag

[marcriera]

What you mention about formatting is very interesting, and I have been doing some research on how it could be better handled in a potential Markdown-based documentation.

Markdown is a format that was designed to easily write in plain text with the ability to convert it easily to HTML and other formats. It supports basic formatting using characters which convey visual meaning instead of the unattractive HTML tags. However, despite its many extensions, Markdown was not designed to replace HTML, so there are elements (such as divs, crucial in HTML styling) which are impossible to recreate without resorting to inline HTML tags.

Ideally, we should have a clean, super user-readable file without any kind of non-Markdown, but given that Markdown was not conceived to replace HTML files, it seems very unlikely. Even basic formatting elements such as subindex or superindex are not supported in most implementations (including Blackfriday, on which Hugo relies). However, I think that the proposed raw data is readable enough for what we want to achieve and should not get overpopulated with HTML tags or Hugo shortcodes, as they are only needed around very specific fragments. The shortcode for warnings and notices, which is the most convoluted one, can definitely be simplified to something cleaner and more Markdown-like. Check this.

As for the Translate attribute, I think it is great and it could be added easily to some parts of the documentation such as command names and argument specifications. It is most useful when translating HTML files directly (not the case here, as markdown files would be translated), but it is also used by machine translation systems to prevent literal translations (which could affect us).

The documentation upgrade would simplify raw files up to the point of them being user-readable, but nonetheless documentation on its structure should be added. This would include a detailed explanation of any specific shortcodes and tags, good Markdown writing, how to carry out translations (what to translate and what to keep unchanged) and how to generate the HTML files and include them in the openBVE release process.

[marcriera]

This has been solved for some time by https://github.com/leezer3/OpenBVE-Documentation-Hugo. Closing the issue.