"Collated Derivative" = Summary dataset produced by stitching together multiple other datasets results.
Note UK summary changes to more standardised format - currently this outputs a dated file in the old archival style. With this change it will just output united-kingdom/collated/rt.csv
This task also brings a lot of tidy up in areas that needed expanding with the collated derivatives - entities are broken out to individual files, lists are in their own subfolder too.
Also switched to using
if (!exists("add_uk", mode = "function")) source(here::here("R", "utils.R"))
at the top of files to list required includes - this should ensure they are only pulled in once but guarantees they are available in that file. By starting to do this in each file it's possible to pick and choose which bits you use when cobbling random test scripts together and ensures each piece can be used in more isolation.
e.g.
source("R/collate-derivative.R") doesn't require to run all the source(x) from the top of run-region-updates before calling it
This is handy for testing / debugging as you can call the script much like it would be in the bigger suite
source("R/collate-derivative.R") # get the script to collate derivatives
source("data/runtime/config.R") # load the dataverse config (optional, allows testing of publication)
source("R/lists/collated-derivative-list.R") # load the list of defined collated derivatives
futile.logger::flog.threshold(futile.logger::TRACE) # set your log level for debugging
collate_derivative(COLLATED_DERIVATIVES[[1]]) # call the function the same way it is from run-region-updates.R and check it works
closes #116
"Collated Derivative" = Summary dataset produced by stitching together multiple other datasets results.
Note UK summary changes to more standardised format - currently this outputs a dated file in the old archival style. With this change it will just output united-kingdom/collated/rt.csv
This includes publishing the collated dataset to DV - https://demo.dataverse.org/dataset.xhtml?persistentId=doi:10.70122/FK2/6GNVT2
This task also brings a lot of tidy up in areas that needed expanding with the collated derivatives - entities are broken out to individual files, lists are in their own subfolder too.
Also switched to using
at the top of files to list required includes - this should ensure they are only pulled in once but guarantees they are available in that file. By starting to do this in each file it's possible to pick and choose which bits you use when cobbling random test scripts together and ensures each piece can be used in more isolation. e.g.
source("R/collate-derivative.R")
doesn't require to run all the source(x) from the top of run-region-updates before calling itThis is handy for testing / debugging as you can call the script much like it would be in the bigger suite