Augment Airtable utils to create Jenkins Jobs

[robons]

Currently the airtable-utils project automatically scans a table contained in airtables (the table is managed by the project's BAs) and automatically creates Jenkins builds for each dataset that it finds (it also creates folders inside relevant git repos too).

Airtable-utils automatically creates Jenkins jobs for each dataset against the deprecated https://ci.floop.org Jenkins instance. We don't want to stop it doing this just yet, but we want airtable-utils to additionally create new csvwlib-style builds on the newer https://ci.ukstats.dev Jenkins server.

It should create two builds per dataset:

• The CSV-W build process
  • This should be triggered upon successful commit to the dataset's folder inside the git repo.
  • The pipeline should run the buildcsvw function.
• The CSV-W -> PMD upload pipeline
  • this should be automatically triggered by a successful build of the first step
  • The pipeline should run the (currently incomplete) csvw2pmd function.
  • ~New parameters should be added somewhere against the airtable table to allow overriding the following csvw2pmd parameters: csvwUrl (if set, the first CSV-W build job should not be created and csvwJobName should be null), graphUriBase and resourcesUriBase~.

This could be an issue for the new devops person?

Don't let perfect be the enemy of good on this one, we need something now and not in 5 months.

[robons]

Documentation on how to switch to using the csvcubed upload pipelines can be found here.