covidatlas / li

Next-generation serverless crawler for COVID-19 data
Apache License 2.0
57 stars 33 forks source link

Develop a schema for scraper output #418

Closed jzohrab closed 4 years ago

jzohrab commented 4 years ago

Original issue https://github.com/covidatlas/coronadatascraper/issues/779, transferred here on Monday Apr 13, 2020 at 02:54 GMT


We have schema definitions for scrapers, but now what they output. We should enforce things like administrative levels and measured quantities (e.g. cases, deaths, etc.).

Along with this, I propose we write a scraper skeleton, with a list of data requirements and pseudo-code (or a code template of some sort) showing what needs to be done and when.

Schema should be validated in yarn test.

@jzohrab @shaperilio