As a PM, I want to review core datasets and make sure all of them are up to date so that I'm sure in their quality
As a PM, I want to re-run and make sure scrapers/scripts are working fine so that I can update data anytime
As a PM, I want to translate scrapers/scripts into dataflows, where possible, so that we use our tools to get the data, plus we can easily update them
As a PM, I want to review READMEs of the core dataset and update where necessary, so that I (and users) are sure that dataset descriptions are accurate enough
Acceptance Criteria
[x] We have the latest data for all core datasets
[ ] We have all of the non-complex scripts working OK (unless the source is broken)
[ ] We have the missing sources and complex scripts fixed up
[x] READMEs are up to date
[x] We use dataflows to get the data
[ ] Automated by Travis
Tasks
[x] List all datasets
[x] Find and fix the non-complex scripts that are not currently working and review READMEs #256 #276
[x] Fix scripts that are complex and have non-common errors #292
[x] translate scripts to dataflows #257
[ ] Fix/refactor more simple scripts addendum #294
[ ] Fix the broken source datasets #291
[ ] Fix scripts that require further analysis and debugging #293
As a PM, I want to review core datasets and make sure all of them are up to date so that I'm sure in their quality
As a PM, I want to re-run and make sure scrapers/scripts are working fine so that I can update data anytime
As a PM, I want to translate scrapers/scripts into dataflows, where possible, so that we use our tools to get the data, plus we can easily update them
As a PM, I want to review READMEs of the core dataset and update where necessary, so that I (and users) are sure that dataset descriptions are accurate enough
Acceptance Criteria
Tasks