epiforecasts / covidregionaldata

An interface to subnational and national level COVID-19 data. For all countries supported, this includes a daily time-series of cases. Wherever available we also provide data on deaths, hospitalisations, and tests. National level data is also supported using a range of data sources as well as linelist data and links to intervention data sets.
https://epiforecasts.io/covidregionaldata/
Other
37 stars 18 forks source link

Automated creating github workflow files for countries #357

Closed joseph-palmer closed 3 years ago

joseph-palmer commented 3 years ago

I noticed how often I forget to write the github yaml files so I've written code to automate it. Hopefully this will save both contributors and us some time.

This PR adds a test which checks there is a github yaml file for each country. If one is not found it creates it from a template with the correct countries details in.

Perhapse need to add a not on cran tag so it only get ran when running devtools::test()?

If approved we will need to update the wiki that users no longer need to add a file manually.

github-actions[bot] commented 3 years ago

👋 Thanks for opening this pull request! Can you please run through the following checklist before requesting review (ticking as complete or if not relevant).

Thank you again for the contribution. If making large scale changes consider using our pre-commit hooks (see the contributing guide) to more easily comply with our guidelines.

joseph-palmer commented 3 years ago

This latest push adds a function make_new_data_source() which makes either a national or subnational class for a given source. So if you wanted to make a new country called Russia, you would call make_new_data_source("Russia") and it would make a template for Russia and make a github action file. You can also make a national data source by setting the type argument to national. This will replace DataClass with CountryDataClass in the inherits from section and change the family to national`.

The test is as before but no longer makes a github action file, just checks its there and tells the user how to make it if it throws an error. The issue is this will only work when running devtools::test() and not on check. This is because the .github files are not part of the R package, just the repo.

I can see two ways around this. 1) drop the test and instead make a github action that will do this check. 2) amend the test so it only runs when called with devtools::test() and not on check. I think solution 1 is better as it keeps repo stuff seperate from package stuff.

TODO before merging in:

joseph-palmer commented 3 years ago

Ive adde the github action to check for the yaml file. I did this by building the package and running a script in inst called test-github_workflow_present.R which I adapted from the previous test to throw an error with an informative message about the mising source. Not sure if this should be put in the .Rbuildignore.

This works but I have some what frankensteined this action together working from another action. If anyone more familiar with actions can suggests improvements I'd be very grateful :)

RichardMN commented 3 years ago

Sorry to arrive to the conversation late. I appreciate and agree with the usefulness of the idea of providing yaml files for the check. Could the check be something which is done in a workflow itself, possibly without even having an R script? This feels like something which doesn't need to get into R, or which might just use R to customize a template as necessary (if we can't find awk or sed).

joseph-palmer commented 3 years ago

github actions are not my strong point, this just worked but there is almost certainly a better way of writing this action, although I couldn't figure it out. If you can think of a better way please spawn a branch off of this one with the action for comparison and if happy we can merge into this :)