chore: add a tool to check links [POC]

tbouffard commented 10 months ago

WIP Covers #430.

Use htmltest: https://github.com/wjdp/htmltest

Configuration of the preview script use to build the site that is checked. This is why we are not using the production build for the checks:

only build the supported version: out-of-support content is not updated
no "edit this page": generate a lot of links to GitHub, this is an Antora feature, we don't want to test that
force usage of the production navbar to have the content as close as possible of the production site
use a dedicated "type" option to produce convenient HTML files for URL checks (HTML extension for local file browsing, no redirect pages)

The GH workflow enables the real use of Antora Atlas to resolve links to pages not built in the preview (links to pages related to unsupported versions). This may consume extra bandwidth in our Netlify plan.

Remaining tasks

[x] explain how to install locally in the readme or in a dedicated doc file. Including links resolutions to pages of unsupported version (with Antora Atlas)
[x] upload the artifact of the build site
[x] upload the artifact of htmltest logs
[ ] review the htmltest configuration to ignore dependencies pages as they are generated and we cannot changes wrong external links. Try to use the IgnoreURLs option - IgnoreDirs cannot be used as the dependencies pages are located in the ROOT module at the root of the directory.
[ ] how do we handle refcache update? commit the cache periodically by creating a PR after the execution ?
[ ] build on schedule: do we send a slack notification on error?
[ ] remove extra changes used during development
[ ] plan fixes in the documentation content repository

Example of preview commands to generate the site to check locally

Single component version

$./build-preview-dev.bash --hide-edit-page-links true --force-production-navbar true --type links-check --local-sources --component bonita --branch 2021.2
$ ./build-preview-dev.bash --hide-edit-page-links true --force-production-navbar true --type links-check --local-sources --component cloud --branch master

Note:

it is possible to activate Atlas reference resolution by installing Antora 3.2 alpha
in this case, the link resolution is done with the production site if the page is not available locally
this could help to test smaller part of the site by also checking some part of the production site (consume more bandwith for the Netlify site)

Build whole site (include bonita out of support version)

$ ./build-preview-dev.bash --hide-edit-page-links true --force-production-navbar true --type links-check --local-sources --use-all-repositories true

Command to only build supported versions The real check requires to enable Antora Atlas. Otherwise, some cross references are not resolved like WARN: missing ref for bonita 2021.1 - some content have xref to this version

./build-preview-dev.bash \
  --hide-edit-page-links true \
  --force-production-navbar \
  --type links-check \
  --use-multi-repositories \
  --component-with-branches bcd:3.6,4.0 \
  --component-with-branches bonita:archives,2021.2,2022.1,2022.2,2023.1,2023.2 \
  --component-with-branches central:1.0 \
  --component-with-branches cloud:master \
  --component-with-branches labs:master \
  --component-with-branches test-toolkit:1.0,2.0 \
  --local-sources

Output

analysis of supported versions (without bonita 2021.1) 2023-09-01
✘✘✘ failed in 11m27.1693513s
987 errors in 3135 documents

Installation options

$ curl https://htmltest.wjdp.uk | bash -s -- -h
bash: download go binaries for wjdp/htmltest

Usage: bash [-b] bindir [-d] [tag]
  -b sets bindir or installation directory, Defaults to ./bin
  -d turns on debug logging
   [tag] is a tag from
   https://github.com/wjdp/htmltest/releases
   If tag is missing, then the latest will be used.

github-actions[bot] commented 10 months ago

🎊 PR Preview c7f6a8f730d191297abaf776e30ba3497362a4ae has been successfully built and deployed to https://bonitasoft-bonita-documentation-site-test-pr-618.surge.sh

:clock1: Build time: 0.013s

_{🤖 By surge-preview}

github-actions[bot] commented 10 months ago

🎊 PR Preview 944a9bb937d99a98539bb2b5a1632686b4a3394c has been successfully built and deployed to https://bonitasoft-bonita-documentation-site-site-pr-618.surge.sh

:clock1: Build time: 0.013s

_{🤖 By surge-preview}