Demo new documentation platform

[lewisdaly]

Goal:

As a new community member I want to read beautiful and easy to read docs so that it's easy to learn about mojaloop

This is an investigation ticket:

Background:

GitBook has now moved to a fully hosted service (with free OSS plans: https://www.gitbook.com/pricing)

This means that our current documentation pipeline is a little out of date, and will no longer receive updates or improvements. What's more, the hosted version features:

• Prettier documentation
• Automatic export to PDF (I remember @millerabel mentioned this would be nice)
• 2 way syncing with github (we can edit docs inside of GitBook and it will push the changes upstream to github, which is great for simple fixes such as typos or links)

Acceptance Criteria:

• [ ] A decision about whether or not we should move to hosted gitbook or perhaps another service

Complexity: <High|Medium|Low> > A short comment to remind the reason for the rating

Uncertainty: <High|Medium|Low> > A short comment to remind the reason for the rating

---

Tasks:

• [ ] TBD [ @? ]

Done

• [ ] Acceptance Criteria pass
• [ ] Designs are up-to date
• [ ] Unit Tests pass
• [ ] Integration Tests pass
• [ ] Code Style & Coverage meets standards
• [ ] Changes made to config (default.json) are broadcast to team and follow-up tasks added to update helm charts and other deployment config.
• [ ] TBD

Pull Requests:

• [ ] TBD

Follow-up:

• N/A

Dependencies:

• N/A

Accountability:

• Owner: TBC
• QA/Review: TBC

[elnyry-sam-k]

@lewisdaly do you think this is a good time to bring it up with @simeonoriko , Megan and Paula?

[lewisdaly]

Yeah I think Megan already followed this up, and it looked like they could put us on their free plan. I think we just need to hve a conversation to see if it's something that everyone wants, and we ac

The main thing for me is (1) the (greatly) improved design, and (2) continued updates into the future, since we are using the legacy gitbooks that no longer gets updates.

[elnyry-sam-k]

Thanks @lewisdaly , then it does make sense to move to the (free) plan.

[kjw000]

agree. I will reach out to Gitbooks to confirm what needs to be done there. We need to move off the legacy system. Once I get the information - I will schedule a meting to figure out the effort of upgrading?

[millerabel]

We certainly need to be on a supported and updated platform! Hosted services, if free for open source, are a good option. Let’s evaluate whether the style sheet support can be added by us and so maintained directly by us as part of publication, if we updated to the latest platform. Or alternatively what benefits come from moving to a hosted service.

— Miller

On Nov 1, 2020, at 6:15 PM, Lewis Daly notifications@github.com wrote:

 Yeah I think Megan already followed this up, and it looked like they could put us on their free plan. I think we just need to hve a conversation to see if it's something that everyone wants, and we ac

The main thing for me is (1) the (greatly) improved design, and (2) continued updates into the future, since we are using the legacy gitbooks that no longer gets updates.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[lewisdaly]

Some more context from discussions with Miguel:

• the Hosted gitbook doesn't look like it supports all the plugins we need: mainly swagger, plantuml-svg & editlink

  Note from Lewis - I find the plantuml diagrams often freezing up my browser and don't allow zooming or scrolling, so perhaps there is a better alternative anyway

• It's worth looking at other options (self-hosted or otherwise) that may suit us better if we are going to do some refactoring work, for example:
  • https://vuepress.vuejs.org/ (which has a good set of plugins, and is compatible with our workflow)
  • http://gatsbyjs.com/

[lewisdaly]

Here's a sample of a good vuepress site: https://zircleui.github.io/docs/

[lewisdaly]

https://docsify.js.org/#/ also looks good

[kjw000]

very slick - I have reached out to gitbooks Mon and today without any response. Customer support is not good (of course we are on the free plan). Perhaps we can create a quick comparison of the the top ones and we need to make sure they allow for free open source licenses, have plug in support, 2 way sync with mark down and can easily import images, etc.

[lewisdaly]

I spent a bit of time seeing if I can reproduce the functionality we need from GitBooks with VuePress:

Here's a quick (and nasty) example of turning the PISP repo into it's own mini site:

https://docs.mojaloop.io/pisp/

It has most of what we need (and maybe even more...). The main things to resolve are:

• swagger site generator - I can't find a plugin, but this site is doing exactly that: https://docs.zaikio.com/api/directory/directory.html
• plantuml/svg renderer: - for consistency between our github markdown and documentation sites, I would favour exporting to .svg using a git precommit hook or something similar. That way we don't have the following popping up in our github: https://github.com/mojaloop/documentation/blob/26a8382ee65ec672c9ee15625171d6829220596e/mojaloop-technical-overview/quoting-service/qs-post-bulk-quotes.md

[lewisdaly]

Here's a little comparison table I've started:

Tool	Hosted/Self Hosted	Community Support	Hackable	Plugins	plantuml	swagger	SEO
Legacy Gitbooks	Self Hosted	NO	YES	YES	YES	YES	?
Gitbooks Hosted	Hosted	NO	NO	NO	?	NO	?
Vuepress	Self Hosted	YES (vue)	YES	YES	NO	NO	Good
GatsbyJs (Docz)	Self Hosted	YES (react)	YES	YES	?	?	?
Docsify	Self Hosted	YES	YES	YES	?	?	Bad - no static site
Docasaurus	Self Hosted	YES (react)	YES	YES (v2 only)	?	?	Good

Additional Notes:

• docsify looks a little lightweight to me
• Vuepress has a good level of configurability, and seems to be the open source sucessor to Legacy Gitbooks
• Docz looks pretty impressive, it uses .mdx (markdown + JSX) to render documentation. Moving to this tool will be more work than moving to Vuepress.
• Docasaurus is another static generator that uses react. They are currently working on v2, which has plugins, but uses an mdx file format

Here's my $0.02:

• Hosted Gitbooks probably isn't the right call since it isn't as configurable as what we have now
• The 3 main static generators look pretty good: Vuepress, Docz and Docasaurus, and each have similar levels of community support
• Migration to Vuepress will be the easiest (at least in my few hours of playing around with them)
• I don't think we want .mdx files (which Docz and Docasaurus use), since it's easier to have plain markdown that renders similarly in Github as well as our documentation site

[elnyry-sam-k]

Thanks @lewisdaly , without this table, I would've voted for hosted Gitbooks version, but until thats standardized, looks like Vuepress is the candidate, with available information..

[kjw000]

Thanks for the table @lewisdaly - do you think we can do a pilot with vuepress? is swagger plugin a non-started?

[lewisdaly]

Thanks Kim, yeah I think we can do a small portion of our docs in vuepress and get a feeling for the effort involved here.

I've found an even better alternative for the swagger plugin using 'custom components' in vuepress. See here for an example or the draft 3rd party api. I think it's much better than what we have at the moment.

[lewisdaly]

Heres a quick (and slightly nasty) 2 hour attempt at moving from GitBooks to Vuepress:

https://vessels-tech.github.io/documentation/home.html

The main cool thing to note is the auto-rendering swagger docs:

https://vessels-tech.github.io/documentation/api/mojaloop-api-specification.html

[elnyry-sam-k]

Looking good, @lewisdaly .. I think at this point, I can vote for Vuepress..

[kjw000]

This looks great. is there anything that is not working as expected? It looks like the links to the internal pages have some issues, but this can be fixed.

[millerabel]

This is a good demonstration of the formatting available using Vuepress and the ability to auto generate API doc from the specification has potential. Nice. Would we be able to generate sequence diagrams from the puml? It would be interesting to compare the options to manage SVGs. Generate and cache them from the modified GitHub primary sources? — during the PR process or post-merge compilation or just-in-time when requested?

I will point out that the generated documentation for FSPIOP 1.0 is atrocious! Look at POST transfers (what the heck is a “next ledger”??) and PUT transfer/{id} (the sentence is nonsense, it omits important words and teaches nothing about what the callback actually means, what a DFSP should do when receiving it, or that it might not be generated before a timeout occurs). Who issues the POST and who issues the PUT? Are they related? And where does this request and response fit into the protocol sequence?

We need a rigorous copy editing pass over the documentation and the API Specification. And the generated API doc needs more cross-referencing to key concepts and terminology definitions. And callbacks should refer back to the request that caused them. These facets can be provided automatically with minimal markup in the source documents.

Remember the advice Kurt Vonnegut gave to his writing students... “Pity the Reader.”

— Miller

On Nov 13, 2020, at 12:02 AM, Lewis Daly notifications@github.com wrote:

 Heres a quick (and slightly nasty) 2 hour attempt at moving from GitBooks to Vuepress:

https://vessels-tech.github.io/documentation/home.html

The main cool thing to note is the auto-rendering swagger docs:

https://vessels-tech.github.io/documentation/api/mojaloop-api-specification.html

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[kjw000]

Now that I had time to review in more detail, seems like there are several differences. Let us discuss in our Tuesday DevOps meeting to figure out if this is configuration or a feature issue.

1) Navigation tree is quite different. https://docs.mojaloop.io/documentation/ Mojaloop background (core scenarios and L1P missing from: https://vessels-tech.github.io/documentation/deployment-guide/

2) No Table of Contents on Pages https://docs.mojaloop.io/documentation/deployment-guide/ vs. https://vessels-tech.github.io/documentation/deployment-guide/

3) Hot links on pages do not work https://vessels-tech.github.io/documentation/deployment-guide/

4) Some edits back to github result in 404 error https://vessels-tech.github.io/documentation/getting_started.html (bottom of any page)

5) Some of the levels did not import correctly. For example the glossary of terms is now concepts is now at the highest level https://vessels-tech.github.io/documentation/glossary.html https://docs.mojaloop.io/documentation/glossary.html

I am sure others. thanks.

[lewisdaly]

Thanks Kim and Miller for your feedback.

Like I said, it's quick and nasty, and I haven't yet had a chance to recreate some of the other features you mentioned @kjw000. My main goal was to answer the questions: "Is this feasible with Vuepress?" and "What is the effort involved?

I'm looking forward to discussing more in person.

@millerabel, you raise a great point about the auto generated documentation from swagger files- let's follow this up separately, as it's been on my mind as well. Another issue I see with it is the sheer amount of noise the header parameters make in the rendered documentation.

Regarding the .svg files, this is something I've been talking to @mdebarros about, and we were thinking of using a simple git commit hook to rerender changes to the .svg files, and then another circleci step that makes sure that checks for outdated .svg files just in case.

We're currently testing this out on the PISP repo, and it's an approach I see working. One of the advantages is that it makes the documentation easy to read and edit both on the rendered docs site as well as the .md sources in github. We have also tried 'just in time' .puml server rendering in the past (in the PISP repo), but I've never been impressed with the reliability of that approach.

[lewisdaly]

Blocked by: https://github.com/mojaloop/documentation/pull/278, rendering svgs in git commit hooks. Once we have this done, we will depend much less on specific gitbooks tooling.

[lewisdaly]

@kjw000 @mdebarros @elnyry-sam-k

Ok. I think we're about 80% of the way there:

https://vessels-tech.github.io/documentation/ btw the link has changed from above

Here's what's on my plate next:

• fix technology overview section - some screens not loading because of inconsistencies in .md file structures
• remove puml tags in favour of svgs (I will follow up in a different PR to minimize the size of this one)
• minor styling tweaks
• dead link scanning automation
• table of contents for mobile view (currently it just disappears)

[elnyry-sam-k]

Looks great, Lewis! Thanks..

[kjw000]

Looks great however there is one rendering issue that is still odd.

I will send via email.

[godfreykutumela]

Hi @lewisdaly, Is there any pending activity on this issue? if not lets move it to review then close it at the end of the current sprint.

[lewisdaly]

Hey @godfreykutumela let's hold off for now until we can hand off to the new tech writer. From that point, we can create a few followup tickets and close this one.