covidatlas / li

Next-generation serverless crawler for COVID-19 data
Apache License 2.0
57 stars 33 forks source link

Add report monitoring #450

Closed jzohrab closed 4 years ago

jzohrab commented 4 years ago

Original issue https://github.com/covidatlas/coronadatascraper/issues/997, transferred here on Tuesday May 12, 2020 at 15:47 GMT


Description

Monitor generated reports that are publicly available (e.g. at https://coronadatascraper.com/#timeseries-jhu.csv, or https://coronadatascraper.com/data.json) to ensure that:

a) data is available b) data is valid

Why do you need this feature or component?

User impact (ref https://github.com/covidatlas/coronadatascraper/issues/996)

Additional context

Point b) should be handled during report generation, and perhaps the validation rules could be published as well (if it's easy!) for ppl to look at.