search

NYCPlanning / db-equitable-development-tool

Data Repo for the equitable development tool (EDDT)
MIT License
0 stars 0 forks source link

WIP: PUMS data update for 2023 data #320

Closed fvankrieken closed 1 year ago

fvankrieken commented 1 year ago

312

A couple big buckets of changes

  1. data updates. Mostly added excel files, updated datasets.yml as well
  2. Testing/github workflow changes. Largely covered in #321, can mostly ignore for review here
  3. Logic around cleaning column names. There was a lot of duplicated logic around cleaning column names relying on specific years, both for things involving the ACS data and not. I further centralized some logic here where some of this work had previously been done, then tweaked logic in many files to reflect this. There's also now one hardcoded spot of ACS years here, seemed simplest for now to leave the default args in all the indicators/accessors but make it so that it only needs to be updated in one spot

Most recent actual builds

latest s3 outputs

Lastly, theres this file - I really can't find an older equivalent of it. For now I put it and the income adjusted ACS files in the archive folder, just to keep them for posterity

fvankrieken commented 1 year ago

Builds here:

https://github.com/NYCPlanning/db-equitable-development-tool/actions/runs/4546451152 https://github.com/NYCPlanning/db-equitable-development-tool/actions/runs/4546732774 https://github.com/NYCPlanning/db-equitable-development-tool/actions/runs/4546750916

fvankrieken commented 1 year ago

Most recent actual builds https://github.com/NYCPlanning/db-equitable-development-tool/actions/runs/4610789538 https://github.com/NYCPlanning/db-equitable-development-tool/actions/runs/4610318167

latest s3 outputs https://nyc3.digitaloceanspaces.com/edm-publishing/db-eddt/fvk_pums_20230323/2023-04-06

fvankrieken commented 1 year ago

@damonmcc edited the main description of this pr to give better context. Ready for review!