Improving the way the docs are built/hosted

[edmorley]

Currently the Neutrino 8 docs are manually built locally using gitbook and then pushed to the gh-pages branch, where they are served by GitHub over the custom domain https://neutrino.js.org

There are a few problems with this:

• the docs don't auto-deploy - which means someone has to remember to do so, and then spend time building/deploying.
• there's no easy way to have multiple versions of the docs served at once (unless manully scripting the docs build to git checkout every release branch and somehow combine the output; though there's no guarantee the master build deps will work on older branch's docs/ directories).
• the gitbook/gitbook-cli packages are no longer maintained (having been replaced by the new hosted gitbook beta).
• the gitbook-cli package has a massive dependency list that includes npm (since it self-downloads the main gitbook package?!) - and npm has bundledDependencies which yarn doesn't handle well.

We tried switching to the new hosted Gitbook beta in #862, however:

• the two way sync between their internal gitbook metadata store and the GitHub repo is both buggy and unnecessary for our requirements (making all updates though version control makes more sense to me).
• a lot of the build/theme behaviour is auto-magical, not documented and no way to customise.
• custom gitbook plugins are not supported, so there is no way to do things like include markdown from one file in another.
• the new site is much slower.
• support took ages to respond & didn't really answer our questions (though free account, so fair enough).

Wanting simple multi-version docs rules out GitHub pages. I looked into https://readthedocs.org but they don't allow custom themes.

A promising looking option is Netlify, which:

• fully automates deployments.
• can use whatever build toolchain you prefer, meaning we can move away from the deprecated gitbook package.
• supports having multiple versions of the docs (via subdomain per branch).
• supports PR deploy previews and other great features.

Possible options for build tools/themes (using https://www.staticgen.com/ for inspiration):

• MkDocs (themes)
  • https://squidfunk.github.io/mkdocs-material/
• VuePress
• Docusaurus
• Hexo (themes)
  • https://zalando-incubator.github.io/hexo-theme-doc/
• Hugo (themes)
  • https://learn.netlify.com/en/
  • https://docdock.netlify.com/
  • http://themes.gohugo.io/theme/material-docs/
• Gatsby (themes)
• Jekyll (themes)
• Sphinx (themes)
• staying with deprecated gitbook (but I'd really prefer not to)

TODO:

• [X] Register new neutrinojs.org domain (since we don't control the js.org domain and Netlify needs us to use its DNS nameservers for features like deploy previews and simpler subdomain setup).
• [X] Create neutrinojs project on Netlify.
• [X] Add neutrinojs.org as a custom domain to Netlify & adjust nameservers accordingly.
• [X] Once DNS changes propagated, enable Let's Encrypt powered TLS, as well as HSTS.
• [X] Experiment with Netlify prototype using mkdocs + material theme and docs from master.
  • -> this worked well.
• [x] Add a netlify.toml to the release/v8 branch, which builds the v8 docs using gitbook (still), but means we can switch away from github pages as an initial step (will be easier than having two different hosting solutions for Neutrino v8 vs v9 docs).
  • -> #891.
• [x] Once #891 merged, set up the Netlify "production" docs to be built from the release/v8 branch, and check everything looks ok on https://neutrinojs.org.
• [x] Make the old domain be served by Netlify too, by adding as an extra custom domain to Netlify settings, then opening a PR like js-org/js.org#1948 to update the neutrino.js.org DNS to CNAME against Netlify instead of GitHub pages.
  • -> js-org/js.org#2021.
• [x] Pick which static site generator / theme we want to use for master (Neutrino 9).
  • -> mkdocs + material theme.
• [x] Update master to use the new generator/theme and add a netlify.toml with relevant build steps and redirects.
  • -> #898.
• [x] Configure netlify to deploy master as a subdomain of the new domain. (~We may need to contact support to see if there is a way to customise the subdomain name, given that at first glance it always matches the branch name, so we couldn't call it eg next. Though not a blocker if this isn't possible.~ Let's not bother, I think master.neutrinojs.org is fine.)
• [x] Update the release/v8 netlify.toml to make any pages viewed via the old domain redirect to the new domain.
  • -> #903.
• [x] Update all links to point to the new domain (GitHub repo description/NPM package readmes/...).
  • -> #903 for release/v8, #910 for master.
• [x] Once we're happy with the new setup, delete the now unused gh-pages branch and disable the repo GitHub pages setting.

Once Neutrino 9 is released, the Netlify production branch would then be set to either master or release/v9 (depending on when we choose to branch), and release/v8 then be served under a subdomain that we can link to from the table of contents of the Neutrino 9 docs.

[edmorley]

Once #891 merged, set up the Netlify "production" docs to be built from the release/v8 branch, and check everything looks ok on https://neutrinojs.org.

This has now been done.

@mozilla-neutrino/core-contributors (or anyone else :-)), could you check to see if you can see any diffrences (other than the recent typo/... fixes etc that landed on that branch) between these two?

• Old GitHub pages powered site: https://neutrino.js.org/
• New Netlify powered site: https://neutrinojs.org/

Once we're happy the site on Netlify looks fine, I'll get the js.org DNS updated to point the CNAME at Netlify instead of GitHub pages for the old/existing Neutrino domain.

[eliperelman]

Everything seems OK to me.

[edmorley]

Other than future wording/formatting tweaks to the docs on master, we're all done here.

Summary of current state:

• We now have a new home, https://neutrinojs.org
• https://neutrino.js.org HTTP 301 redirects to https://neutrinojs.org
• The docs are entirely served by Netlify - GitHub pages is no longer used at all (gh-pages branch deleted).
• The master branch docs auto-deploy to https://master.neutrinojs.org
• The release/v8 branch docs auto-deploy to https://neutrinojs.org
• The release/v7 branch docs auto-deploy to https://release-v7.neutrinojs.org
• The docs for even older versions are available via GitHub, as they were before. However now consistent release/vN branch names are used (eg: https://github.com/mozilla-neutrino/neutrino-dev/tree/release/v6/docs) instead of docs-vN tags. The old tags have been removed.
• On master we're now building the docs with mkdocs instead of gitbook, and using a new material theme.
• The master, v8 and v7 docs all now correctly link to each other in the sidebar.
• When it comes to releasing Neutrino 9, the Netlify "production" branch setting (that gets deployed to https://neutrinojs.org) will be set to either master or release/v9 (depending on how we choose to handle the release) and the release/v8 branch docs moved to a subdomain like the v7 docs are currently (and various tweaks made like adding v9 sidebar links to v8/v7 etc).

[edmorley]

Netlify now have a GitHub app (vs the old style oauth integration), which I've just updated us to.

One nice extra with this is enhanced GitHub status check reporting :-)