Develop an action for building static websites from datasets

[strogonoff]

A couple of example sites that have scaffolding in their repository:

• https://github.com/paneron/iev-data-subset (builds to iev.demo.paneron.org)
• https://github.com/glossarist/glossarist.org (builds to Glossarist.org)

The idea is to remove the project scaffolding from data repositories, to reduce duplication and make repositories only contain the data. Scaffolding will instead live in separate repository (or, more likely, multiple repositories for different data types). Meanwhile, each data repository will just specify a single action, and pass requisite parameters to it.

We still might need to maintain different scaffolding repositories, but at least there can be one copy for each data type, rather than a copy for each data repository.

The action will receive parameters:

• Which project build scaffolding to use (see below, probably this will be a string containing repository URL)
• Where to deploy the built site (likely for now this will always be S3 bucket ID, along with possibly a CF distribution ID to invalidate cache)

The action would be expected to:

• Fetch the necessary Node project scaffolding (it’d contain package.json, yarn.lock, the src directory, etc.)
• Place it in repository root
• Run a package.json script (most likely, yarn build—we’re using Yarn, although npm build could work as well)
• Expect the result to be built under dist directory in repository root, with index.html as entry point
• Upload the result to the bucket specified, possibly trigger CF cache invalidation

To be confirmed:

• How to specify build scaffolding source? Originally I thought the action will be able to infer the URL based on dataset ID, but now I think we can just give the action a repository URL (especially considering we can have multiple datasets in one repo, and we’re trying to build a single site from all that).

Caveats/possible changes in future:

• A single repository can contain multiple datasets, so from one repository there can theoretically be built multiple sites, or a single site. For now we’re sticking to a single site approach.
• There may be templating involved. Context variables will be passed to the action, and the action will process certain files (possibly, static.config.js) to provide site-specific data in project scaffolding.

Related:

• Dataset preview service we also need to have. This may not be possible to develop using only GHA, as (1) previews may have a stronger requirement to have user authentication, where for a static site hosted on S3 a Lambda function would be best, and (2) the user may want to build a preview on demand, without having changes pushed to production repository (will need to consider how this could be addressed using forks—in that case, preview service spec may change in the future to use this GHA together with a Lambda for auth—but this will involve/complicate how Paneron dataset preview functionality would work).

[strogonoff]

Added a note about possible templating requirement in future.

[ronaldtse]

This is a good idea. Let’s do it.

[strogonoff]

Example scaffolding project: https://github.com/paneron/glossary-site-scaffolding.

As one of the parameters, the action would be given an URL that points to a similar repository. The action would need to fetch the repository and place it in dataset root, per issue description.

(cc @CAMOBAP)

[CAMOBAP]

In progress

[CAMOBAP]

Status:

• build action is ready, test failed because there is no correct scaffolding repo provided
• aws-deploy action implemented as a separate one

https://github.com/paneron/iev-data-subset/pull/5 - ready for review https://github.com/glossarist/glossarist.org/pull/25 - ready for review

@strogonoff I need help with:

• correct scaffolding for iev-data-subset
• change default branch for this repo from feature/first-release to main

[strogonoff]

• Scaffolding is outside of this scope. I’ll update glossary site scaffolding, check that it works and will get back on this
• Filed #6

[CAMOBAP]

@strogonoff according to my understanding this task can be closed, can you confirm?