datahubio / datahub-v2-pm

Project management (issues only)
8 stars 2 forks source link

Refactor top-10 regularly updated datasets to use dataflows #264

Closed zelima closed 5 years ago

zelima commented 5 years ago

As a PM I want to select 15 core datasets, that need to be updated regularly (daily, weekly, monthly, etc...) so that I'm able to write scripts that will be run automatically and my data will always be up to date

As a PM I want to translate 15 scripts for datasets that can be automated into dataflows, so that I'm able to run flows in dpp-server and keep my datasets updated without manual interaction

Acceptance Criteria

Tasks

Analysis

15 datasets from datahub that should be updated on shcedule

id name github_url update period translated to dataflows PR
1 airport-codes https://github.com/datasets/airport-codes dailiy  FALSE https://github.com/datasets/airport-codes/pull/30
2 bond-yields-uk-10y https://github.com/datasets/bond-yields-uk-10y quarterly FALSE https://github.com/datasets/bond-yields-uk-10y/pull/2
3 bond-yields-us-10y https://github.com/datasets/bond-yields-us-10y monthly FALSE https://github.com/datasets/bond-yields-us-10y/pull/2
4 dac-crs-codes https://github.com/datasets/dac-crs-codes   FALSE
5 finance-vix https://github.com/datasets/finance-vix daily FALSE https://github.com/datasets/finance-vix/pull/5
6 household-income-us-historical https://github.com/datasets/household-income-us-historical anually FALSE https://github.com/datasets/household-income-us-historical/pull/2
7 language-codes https://github.com/datasets/language-codes   FALSE
8 natural-gas-prices https://github.com/datasets/natural-gas-prices daily FALSE https://github.com/datasets/natural-gas-prices/pull/3
9 oil-prices https://github.com/datasets/oil-prices daily FALSE https://github.com/datasets/finance-vix/pull/5
10 population-growth-estimates-and-projections https://github.com/datasets/population-growth-estimates-and-projections anually FALSE https://github.com/datasets/population-growth-estimates-and-projections/pull/1
11 football-datasets https://github.com/datasets/football-datasets FALSE
12 s-and-p-500 https://github.com/datasets/s-and-p-500 Monthly FALSE https://github.com/datasets/s-and-p-500/pull/6
13 s-and-p-companies-financials https://github.com/datasets/s-and-p-companies-financials FALSE https://github.com/datasets/s-and-p-companies-financials/pull/4
14 gold-prices https://github.com/datasets/gold-prices FALSE https://github.com/datasets/gold-prices/pull/2
15 exchange-rates https://github.com/datasets/exchange-rates FALSE https://github.com/datasets/exchange-rates/pull/2