google-research / open-covid-19-data

Open source aggregation pipeline for public COVID-19 data, including hospitalization/ICU/ventilator numbers for many countries.
Apache License 2.0
115 stars 69 forks source link

Fix mobility exports file structure + prevent data update from causing the same bugs #104

Closed everettk closed 3 years ago

everettk commented 3 years ago

There were two bugs caused by the data update workflow running src/scripts/export_data.py: 1) Google mobility data was exported into data/exports/google_mobility_reports, when it should be exported into a subdirectory named data/exports/google_mobility_reports/Regions. 2) The region-specific README files in data/exports/search_trends_symptoms_dataset have a relative link to dataset documentation, and that relative link needs to have one less .. than the relative link in the corresponding READMEs in data/inputs. Copying the READMEs directly from inputs to exports caused this link to break.

I fixed #2 in the search-data-update branch and merged it into master locally, then pushed master, because the diff was too bug for github to create a PR. This PR fixes #1 and then prevents export_data.py from causing the same two bugs by commenting out the function calls that caused them. In order to automate the updates for mobility + search data, the underlying issues here will need to be fixed first.