[epic] Wrangle all the data we need

[rufuspollock]

Motivating Questions

Questions I want answering as a citizen:

• Is this city growing / thriving
  • Population ("Population of London"), Unemployment, GDP (per capita)
• Is this a nice city to live in?
  • Is there crime?
  • Is there air pollution? "Average PM2.5 Levels in London"
  • Transport:
• Is this an affordable city to live in?
  • House prices: "Median Housing Affordability Ratio"
  • Is is an international city
• How is the city performing
  • Is it spending more monney than it has ...
  • Life expectancy at birth ... (bit weird)
  • Community-ness ("Talk to neighbours at least once a week")

As someone running a city

• What is cost of living and how it changing (by percentile)
• How is transport performing in terms of safety and on-timeliness
• Is the city growing or shrinking pop and gdp wise
• Are communities thriving (and where and where not)
• What is happening with crime and where ...

Categories

• Growth
• Health and safety
• Environment
• Transport
• Well-being
• (Population dynamics and community)

Secondary

• Transport
• Homelessness

Categories and indicators for (london) city demo

• Economy: GDP, (Un)Employment (total jobs), Population
• Health and Safety: Crime (Recorded Crime), Health e.g. Life expectancy, Air quality (PM2.5)
• Livability: House prices ("Median Housing Affordability"), Transport (delays), Transport (number of journeys)

Indicator list

Status: source identified, wrangled, reviewed, published

• [x] GDP / GVA - https://data.london.gov.uk/dataset/gross-value-added-and-gross-disposable-household-income - i think we want GVA totals (sheet 2) and per head (next sheet?).
• [x] Unemployment - https://data.london.gov.uk/dataset/unemployment-rate-region - ...
• [x] Population
  • [x] 1961-2015 - https://data.london.gov.uk/dataset/office-national-statistics-ons-population-estimates-borough - Mid year estimates since 1961
  • [x] Projections to 2050 https://data.london.gov.uk/dataset/projections/ - https://data.london.gov.uk/download/projections/b9daabe6-6dd3-4082-adcf-a6b458ef4945/central_trend_2016_base.xlsx
  • [ ] Needs a comment in table schema about being thousands
• [x] Recorded crime - https://data.london.gov.uk/dataset/recorded_crime_rates
  • Github repo: https://github.com/datasets/london-crime
  • Datahub: https://datahub.io/core/london-recorded-crime
  • https://github.com/datasets/london-data/issues/2
• [x] Life expectancy - https://data.london.gov.uk/dataset/life-expectancy-birth-and-age-65-borough
  • Github repo: https://github.com/datasets/london-life-expectancy
  • Datahub: https://datahub.io/core/london-life-expectancy
• [x] Air quality - https://data.london.gov.uk/dataset/london-average-air-quality-levels -
  • Github repo: https://github.com/datasets/london-air-quality
  • Datahub: https://datahub.io/core/london-air-quality
• [x] Median Housing Affordability - https://data.london.gov.uk/dataset/ratio-house-prices-earnings-borough
• [x] Transport delays - https://data.london.gov.uk/dataset/london-underground-performance-reports (key trendske)
• [x] Transport number of journeys - https://data.london.gov.uk/dataset/public-transport-journeys-type-transport

Tasks

• [x] (Svetozar) sort co2-ppm (0.5h)
• [x] (Svetozar) Split out one repo for each dataset e.g. london-air-quality (1h)
• [x] (Svetozar) scrape - https://data.london.gov.uk/dataset/life-expectancy-birth-and-age-65-borough (3h)
• [x] (Stefan) scrape https://data.london.gov.uk/dataset/public-transport-journeys-type-transport (2h)
• [x] (Stefan) scrape https://data.london.gov.uk/dataset/london-underground-performance-reports (4.5h)
• [x] (Stefan) push unemployment to github, write an readme (1.5h)
• [ ] (Svetozar) Push to github health inequality indicators dataset (even if incomplete) (1h)
• [ ] Add frequency in a note to the list above (think about how we add to data package!)

Research

This dataset is already an interesting indicator set:

https://data.london.gov.uk/dataset/london-health-inequalities-strategy-indicators

Healthy life expectancy at birth – male Healthy life expectancy at birth – female Children born with low birth weight School readiness among children Excess weight in children at age 10-11 (year 6) Excess mortality in adults with serious mental illness Suicide Mortality caused by Particulate Matter (PM2.5) Employment Feeling of belonging to a community (provisional) HIV late diagnosis People diagnosed with TB Adults walking or cycling for two periods of ten minutes each day Smoking

[anuveyatsu]

@stefanbole can you please comment here with what's left?

[stefanbole]

@anuveyatsu They are all done, except for health inequality indicator dataset, we do have life expectancies do we need more?

[rufuspollock]

@stefanbole @anuveyatsu can you clarify this and close or do the work to close :smile:

[stefanbole]

@rufuspollock, closing this, it's done the health inequality indicator dataset was just discussed.

[rufuspollock]

@stefanbole so can you close it in the appropriate way as per https://handbook.datopian.com/guide/#closing-issues

[stefanbole]

FIXED: The last dataset was finished with fixing the London population repetition in data "https://github.com/datasets/london-population/commit/374ba6bc71b95fc39affa96b55f250d60e696ff0" the staging website was datahub.io.