Add automated link checks

[StevenMaude]

Looking around at other organisations using a "docs as code" approach, a link checker is a fairly standard piece of validation.

There may be other software around, but lychee looks like it could be a good contender. lychee's README also has a list of other competing software.

There may be occasionally private links that we have to skip over. These should be few in number.

[StevenMaude]

lychee report run on content as of 9a3863676387d0b71237e43e69fb03af3a3d375b

Summary

Status	Count
🔍 Total	434
✅ Successful	409
⏳ Timeouts	0
🔀 Redirected	0
👻 Excluded	0
❓ Unknown	0
🚫 Errors	25

Errors per input

Errors in docs/dataset-definition.md

• file:///github/workspace/docs/data-builder-intro: Failed: Cannot find file

Errors in overrides/main.html

• https://fonts.gstatic.com/: Failed: Network error

Errors in docs/actions-pipelines.md

• https://github.com/opensafely/server-instructions/blob/master/docs/Server-side%20how-to.md: Failed: Network error

Errors in docs/study-def-variables.md

• https://www.who.int/classifications/icd/COVID-19-coding-icd10.pdf: Failed: Network error

Errors in INSTALL.md

• https://github.com/opensafely/documentation/network/updates: Failed: Network error

Errors in docs/dataset-ecds.md

• https://github.com/opensafely/rapid-reports/blob/master/notebooks/ethnicity-codes.ipynb: Failed: Network error
• https://datadictionary.nhs.uk/data_sets/cds_v6-2/cds_v6-2-1_type011-_emergency_care_cds.html: Failed: Network error

Errors in docs/study-def-tricks.md

• https://github.com/opensafely/covid-vaccine-effectiveness-research/blob/be747894e1fadf525e391c01477a8ac532613b42/analysis/R/data_process.R#L229: Failed: Network error

Errors in docs/dataset-cpns.md

• https://github.com/opensafely/rapid-reports/blob/master/notebooks/ethnicity-codes.ipynb: Failed: Network error
• https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/882565/Technical_Summary_PHE_Data_Series_COVID-19_Deaths_20200429.pdf: Failed: Network error

Errors in docs/releasing-files.md

• https://www.tandfonline.com/doi/pdf/10.1080/07350015.1989.10509729?needAccess=true: Failed: Network error

Errors in docs/protocol.md

• http://www.encepp.eu/: Failed: Network error

Errors in docs/getting-started.md

• https://github.com/opensafely/research-template/generate: Failed: Network error

Errors in docs/index.md

• file:///github/workspace/docs/workflow: Failed: Cannot find file
• file:///github/workspace/docs/actions-pipelines: Failed: Cannot find file

Errors in DEPLOY.md

• https://developers.cloudflare.com/pages/platform/redirects: Failed: Network error

Errors in docs/dataset-apc.md

• https://www.closer.ac.uk/wp-content/uploads/CLOSER-resource-Understanding-HES.pdf: Failed: Network error
• https://github.com/opensafely/rapid-reports/blob/master/notebooks/ethnicity-codes.ipynb: Failed: Network error

Errors in docs/install-macos.md

• file:///github/workspace/opensafely-cli: Failed: Cannot find file

Notes

• I've only run this twice and got the same number of errors.
• If actually using this to check links on pull request, it's probably worth using the retry option (which is enabled by default, and set to 3).
• The link check took roughly 10 seconds, even in a GitHub Actions runner :racing_car: — so little overhead on a pull request run.

Kinds of error

• Some links are genuinely broken, e.g. https://www.closer.ac.uk/wp-content/uploads/CLOSER-resource-Understanding-HES.pdf
• Some failures are due to use of shorter links that don't refer to .md files. For example, the file:///github/workspace/opensafely-cli failure. We could just make these .md.
• https://fonts.gstatic.com is a preconnect link, but gives a 404 error if you visit it.
• Some links work outside of the Actions run; sites may be blocking GitHub's IP address, e.g. the Cloudflare link.
• Some GitHub links require a login, e.g. https://github.com/opensafely/research-template/generate
• Some GitHub links are to private repositories.
• There were also three email address failures that I've removed. But they are valid email addresses: for example, bennett@phc.ox.ac.uk. It could be some firewall restriction within GitHub's runner. Therefore the number of errors listed is lower than in the count.

[StevenMaude]

This has found some genuinely broken links as listed in #709.

I think this approach is useful. The limitation, as mentioned at the outset, is that a small percentage (about 3%) of links would need periodic manual review.

It might also be one to use for other sites we maintain.

[StevenMaude]

Things to do if implementing:

• [x] #709
• [x] #719
• [x] Disable checks for email addresses, because they don't seem to work within GitHub's runner --exclude-mail
• [x] Use --require-https — this flags HTTP links that are available as HTTPS
• [ ] ~Consider --include-verbatim to check in pre and code blocks (default behaviour in 0.9.0; need to add this in a future release)~
• [x] Curate a list of inaccessible links to exclude

[StevenMaude]

With an exclude file in my own fork, we're now at:

Status	Count
🔍 Total	452
✅ Successful	419
⏳ Timeouts	0
🔀 Redirected	0
👻 Excluded	33
❓ Unknown	0
🚫 Errors	0