update 0610 and 1721 ACS data for 2022 build - Githubissues

NYCPlanning / db-factfinder

data ETL for population fact finder (decennial + acs)

https://nycplanning.github.io/db-factfinder/factfinder/

MIT License

2 stars 3 forks source link

update 0610 and 1721 ACS data for 2022 build #252

Closed damonmcc closed 1 year ago

damonmcc commented 1 year ago

changes

many changes are related to previous manual data update for 2020
replace use of poetry process with requirements.txt
generalize update action/script so it isn't limited to 2020
add data for 2017 - 2021 (aka 2021) ACS data for the manual update
simplify dev container
fix test_metadata.py
update .gitignore and remove ad-hoc .gitignore approaches in init files
add example.env file

notes

Manual update run that generated 2021 ACS data
- compared several variables in output to those in Population Team's excel file and they align
Manual update run that generated new 2010 ACS data
- compared to old data in edm-publishing and new values are 1.2542905 higher than old values
many changes here are from Andrew, Jingyi, and Sasha and were used to generate 2020 data. the last commit from their work was on August 3, 2022
previous 2010 data has been archived in edm-publishing by placing it in a 3-1-22 folder next to the new output. there was already a 10-6-21 folder

damonmcc commented 1 year ago

cc @fvankrieken for visibility on a recent not-small PR