Automating the Propagation of Updates

[bryanhanson]

This is related to #11. Also, see discussion in comments to https://github.com/r-hyperspec/pkg-skeleton/issues/1 .

In addition to automating the GHA workflows, we could also automate the updating of the vignette helper files and files like the .gitignore (which could use some standardization across packages).

[bryanhanson]

I will (soon) start a list of files that could be subject to automated updating.

[bryanhanson]

Here's an incomplete list of files that could be subject to some sort of automatic updating. To be included here, the file should be:

1. Used by most or all packages in r-hyperspec.
2. Be usable as is w/o local modifications.
3. Probably reside in pkg-skeleton, and any modifications should be made there and deployed from there.

In addition, these files should be marked at the top with language like this (would need to comment the message in a an appropriate way depending upon the language/use):

# Original file resides in r-hyperspec/pkg-skelton Edit there and it will be automatically deployed.

• Files at the top level:
  • .gitignore (this file varies a lot across repos and needs to be standardized)
  • .Rbuildignore (this file varies a lot across repos and needs to be standardized)
  • CONTRIBUTING.md (could be included if it were re-worded so that no changes were needed; meaning, make it generic for a package in the r-hyperspec ecosystem).
  • LICENSE (not currently present in pkg-skeleton)
  • codecov.yml

• Files in .github/workflows

  • R-CMD-check.yaml
  • drat--insert-package.yaml (this file is not currently present in pkg-skeleton)
  • pkgdown.yaml
  • test-coverage.yaml

• Files related to vignettes (note: none of these are currently present in pkg-skeleton; if we want these we need to add a minimal vignette that will build):

  • style.css
  • vignette.css
  • vignette-functions.R
  • vignette-default-settings.R
  • elsevier-with-titles.csl

• Files in github-helpers (note: VG had updated these files which I missed, but the new ones still need go be copied):

  • get-skeleton-labels.sh
  • put-skeleton-labels.sh
  • hySpc.skeleton.labels.json (this file is not needed as it is generated by the get script)

---

Finally, here is a recursive directory listing of all the files currently in pkg-skeleton other than the items in .git. . .. .Rbuildignore .git .github .gitignore CONTRIBUTING.md DESCRIPTION LICENSE NEWS.md R README.md _pkgdown.yml codecov.yml github-helpers pkgdown project.Rproj (can be deleted) tests

./.github: . .. .gitignore (can be deleted) workflows

./.github/workflows: . .. R-CMD-check.yaml pkgdown.yaml test-coverage.yaml

./R: . .. demo_function.R package.R unittestdata.R

./github-helpers (note: VG had updated these files which I missed, but the new ones still need go be copied): . .. get-skeleton-labels.sh hySpc.skeleton.labels.json (can be deleted) put-skeleton-labels.sh

./pkgdown: . .. extra.css

./tests: . .. testthat testthat.R

./tests/testthat: . .. test_attached.R

[GegznaV]

@bryanhanson, Nice work preparing the list of files :)

[bryanhanson]

Some thoughts about how to actually do this (notes to self, not final).

• Add the notice to each file to be automatically updated (do this once).
• Upon push to pkg-skeleton, use GHA to:
  • clone each r-hyperspec... repo to the GHA workspace
  • copy each file to be deployed to each r-hyperspec... GHA workspace. I think is is a shell operation because we don't want to change just any file, only selected files, meaning this is not a git merge type operation.
  • commit each r-hyperspec... repo back to its remote

Note that upon push to pkg-skeleton it would be ideal to test if any of the files of interest are changed (a la make) and only then do the subsequent steps. Changes to other files would not trigger the deployment.

[bryanhanson]

There is a working implementation of this idea on branch auto-update in pkg-skeleton. Needs a bit of refinement yet. Look at .github/workflows/auto-update.yaml.

[bryanhanson]

@GegznaV and @cbeleites I believe I have this task accomplished, via https://github.com/r-hyperspec/pkg-skeleton/commit/d9de99a2642e12a3bf3e8f6fd424e94850c9c933 It seems to work under all the conditions I can think of, but before going further perhaps you could take a look and see if there are any flaws in my logic. In the meantime I'm going to think of 1) any additional testing that might be wise and 2) the best way to start using this in a safe manner (I have a tentative plan for this).

[bryanhanson]

A plan to roll out the auto updating of static files:

Note: the working auto-update.yaml is on branch auto-update in pkg-skeleton. Work from there.

• In the repo to be updated, create a branch called auto-update.
• In pkg-skeleton R-CMD-check.yaml add branch auto-update to the list of push triggers (note: since this is one of the files which is automatically deployed, this change only needs to be made once, and then as each repo is updated R-CMD-check.yaml will be updated and trigger the build and check).
• Add the repo to the pkg-skeleton auto-update.yaml workflow package matrix. Make sure that branch auto-update is checked out and then pushed to (so changes in 3 places).
• Push to pkg-skelton which will trigger the workflow. Verify it completes. Investigate any failures. The workflow is pretty verbose so hopefully the nature of any errors is apparent.
• Watch the updated workflows in the repo. Check any failures. Failures may be related to the content of the automatically updated files, or a missing file. If so, edit in pkg-skeleton. Rinse and repeat as needed until it all works.
• Once it all works, merge the auto-update branch into develop and delete the auto-update branch.
• Repeat for additional repos.
• After full testing, branch auto-update of pkg-skeleton should be merged into develop and then deleted.

Warning: once all the repos are listed in the package matrix in pkg-skeleton auto-update.yaml we might hit the GH limit on simultaneous jobs. This needs more investigation.