Fix issues with fetching non-US and non-UK case data

[steven-cd]

The script for populating case data (script/fetch-recorded-data in web-ui) can no longer successfully insert the non-US case data. This seems to because the ECDC data format has changed, and it no longer contains cases or deaths, but cases_weekly and deaths_weekly (possibly related to the ECDC stopping publishing daily data in December). We need to figure out how to account for this.

[sdwfrost]

One thing that puzzled me was that the fetch-recorded-data script uses these links:

const ecdcCasesURL = `https://opendata.ecdc.europa.eu/covid19/casedistribution/json/`
const covidTrackingURL = 'https://covidtracking.com/api/v1/states/daily.json'
const usInterventionsURL = `https://raw.githubusercontent.com/COVID19StatePolicy/SocialDistancing/master/data/USstatesCov19distancingpolicy.csv`
const internationalSchoolClosuresURL = `https://raw.githubusercontent.com/OxCGRT/covid-policy-tracker/master/data/timeseries/c1_schoolclosing.csv`
const internationalRestrictionsOnGatheringsURL = `https://raw.githubusercontent.com/OxCGRT/covid-policy-tracker/master/data/timeseries/c4_restrictionsongatherings.csv`
const internationalStayAtHomeRequirementsURL = `https://raw.githubusercontent.com/OxCGRT/covid-policy-tracker/master/data/timeseries/c6_stayathomerequirements.csv`

while there is an already processed Pandemic Data Lake (https://azure.microsoft.com/en-us/services/open-datasets/catalog/covid-19-data-lake/). That being said, the ECDC data in the Data Lake are currently broken (blank cases/deaths).

It would make sense to add the UK specific coronavirus data here; whether this should be done via the script in the UI or as an Azure Data Factory - we should definitely have our own copy in order to avoid fair use issues and models breaking due to a limit on API calls.

[sdwfrost]

I took a look at what the existing models (outside of the UI) were using. The Basel model uses a variety of data sources:

https://github.com/neherlab/covid19_scenarios/blob/master/data/sources.json.

For the UK, they use Our World in Data - see here, which in turn uses JHU data for cases/deaths plus some other additions/curation. OWID covers lots of countries.

[kavousan]

Renamed as discussed on 17 March 2021 call: covid-policy-modelling/covid-policy-modelling#20 and covid-policy-modelling/covid-policy-modelling#25 sort out the UK data (and we are already ok for US data).