Data tools for loading and processing covid data.
N.B. this is a continuous work in progress, and fairly unstable. I will try and make releases of it as I go but things will probably break in unpredictable ways
library(data.table)
library(dtplyr)
library(tidyverse, warn.conflicts)
here::i_am("common-setup.R")
if (!"ukcovidtools" %in% c(devtools::dev_packages(),rownames(installed.packages()))) {
if (fs::dir_exists("~/Git/uk-covid-datatools/")) {
devtools::load_all("~/Git/uk-covid-datatools/")
} else {
devtools::install_github("terminological/uk-covid-datatools")
}
}
ukcovidtools::setup(here::here("config.yml"))
Where config.yml
looks something like:
default:
librarySource: ~/Git/uk-covid-datatools/
outputDir: ~/covid19/output/
cache: ~/covid19/tmp/
secrets: ~/.covid.yml <see below>
spimSource: s3bucket1 <must match entry in secrets file>
And a secrets file is provided (in example above ~/.covid.yml
) with content like:
sftp:
type: sftp-over-ssh
hostName: <sftp hostname>
ssh: <ssh login@ssh hostname>
sshkey: <ssh key file>
user: <sftp username>
password: <sftp password>
s3bucket1:
type: s3
accesskey: <s3 access key>
secretkey: <s3 secret key>
region: eu-west-2
bucket: <bucket-1 name>
s3bucket2:
type: s3
accesskey: <s3 access key>
secretkey: <s3 secret key>
region: eu-west-2
bucket: <bucket-2 name>
localfiles:
type: local
root: ~/covid19/input/
Code and supporting material for our MedRxiv paper
Code and supporting material for our MedRxiv paper has moved