Build pipeline

[Mischback]

Create an actual build pipeline:

Stages

• [ ] Staging
  • [x] CSS assets -> included in #126
  • [x] introduce logic for a development build -> included in #126
  • [x] provide the actual cache busting setup -> included in #126
    • [x] compilation of stylesheet from theme/mischback/_src/style/style.scss into theme/mischback/static/style.css
    • [x] run sphinx build to generate the HTML output (in .build); this step will place the stylesheet in .build/_static/style.css
    • [x] run PostCSS against .build/_static/style.css
    • [x] using the HTML output as input for purgecss
    • [x] autoprefixer
    • [x] cssnano
    • [x] run PostHTML to perform the actual cache busting
    • [x] run a custom Python script to perform the actual cache busting --> #126

Dependencies

• 120

Reference

• colorizer, which was one of the sidetracks that significantly impacted building this website, has the additional benefit of having implemented a working cache busting in its v2 branch.
• Related: #61

[Mischback]

Build Pipeline

Stage 0: Development

• the most recent code lives in the development branch
• development happens in dedicated feature branches, which are then used in a pull request against the development branch
• PRs are processed with a dedicated workflow through Github Actions:
  • linting (several different linters for the different languages, e.g. black for Python, stylelint and prettier for SCSS/SASS)
  • building the project includes several stages, including compilation of SCSS/SASS and TS/JS sources and running sphinx to actually create the static HTML output. Management of the actual build process is handled by the project's Makefile.
  • validating the created build artifact runs several HTML validation tools against the (generated) output to ensure semantically correct HTML. It does NOT check, if everything is working!
• Please note: The website's images are not part of the repository. Their existence is not checked! The (generated) build artifact of the building step is not ready for deployment!.
• Focus of this stage and the associated workflow is code quality.
• MISSING / TODO:
  • Tests
  • Unit Tests for Python code, especially the utility scripts; try to provide unit tests for the (custom) sphinx extensions aswell
  • Unit Tests for TS/JS code (how to unit test a codebase with target browser?)
  • Integration Test for (compiled) JS code (at least target major browsers)
  • Validation of (compiled) CSS stylesheet

Stage 1: Staging

• the current release is always the last commit of the main branch
• staging is initiated by performing a PR against main
• PRs are processed with a dedicated workflow in Github Actions:
  • linting as described above; this is only a precaution, result should not deviate from development branch.
  • building: This step is a heavy deviation from Stage 0:
  • obtain image sources (tbd)
  • create responsive images
  • create (minified) CSS stylesheet
  • create (minified) JS
  • run sphinx build process
  • cache busting
  • tidy HTML sources
  • validating: This step is a deviation from Stage 0:
  • (still) perform HTML validation as in Stage 0
    • include checks for image sources!
  • perform CSS validation
  • check for moved documents
    • obtain current live sitemap.xml
    • launch build artifact in a webserver (local/Docker) and try to access documents by their live URI
    • This should provide warnings, but does not FAIL the build
  • ensure no 404
    • launch build artifact in a webserver (local/Docker) and scrape the whole site (obviously don't follow external links; they are covered already by sphinx linkcheck and possibly by the HTML validators).
  • Integration Tests / End-to-End Tests
    • Selenium, Playwright

Stage 2: Release

• a workflow is triggered for a commit to main branch; the commit should be a merge commit from a staging branch
• it is an extension to the workflow described in Stage 1:
  • pick up the artifact from building and apply maximum brotli and gzip compression to everything
  • re-upload as the final artifact

[Mischback]

Resources

• run a HTTP server in a Github Actions workflow
  • https://github.com/marketplace/actions/http-server-action