NYCPlanning / db-factfinder

data ETL for population fact finder (decennial + acs)
https://nycplanning.github.io/db-factfinder/factfinder/
MIT License
2 stars 3 forks source link

CDP variable errors for 2015-2019 ACS #85

Closed mgraber closed 3 years ago

mgraber commented 3 years ago
Variable mappings changed between 2018 and 2019. pff_variable base_variable census_variable 2018 definition 2019 definition 2019 census_variable
fb1 pop_4 DP02_0092 Estimate!!PLACE OF BIRTH!!Total population!!Foreign born Estimate!!PLACE OF BIRTH!!Total population!!Native!!Born in Puerto Rico, U.S. Island areas, or born abroad to American parent(s) DP02_0093
ea_bchdh ea_p25pl DP02_0067 Estimate!!EDUCATIONAL ATTAINMENT!!Population 25 years and over!!Bachelor's degree or higher Estimate!!EDUCATIONAL ATTAINMENT!!Population 25 years and over!!High school graduate or higher DP02_0068
pop_4 pop_4 DP02_0086 Estimate!!PLACE OF BIRTH!!Total population Estimate!!RESIDENCE 1 YEAR AGO!!Population 1 year and over!!Abroad DP02_0087
ea_p25pl ea_p25pl DP02_0058 Estimate!!EDUCATIONAL ATTAINMENT!!Population 25 years and over Estimate!!SCHOOL ENROLLMENT!!Population 3 years and over enrolled in school!!College or graduate school DP02_0059

New errors in cases where:

v.pff_variable in self.profile_only_variables 
and geotype in self.aggregated_geography

For example, the following variables from db-community-profiles out_acs.py produces all NULLs except for foreign born in CD 105.

{'pff_variable': 'ea_bchdh',
  'geotype': 'cd',
  'column_mapping': {'z': 'moe_bach_deg', 'p': 'pct_bach_deg'}},
{'pff_variable': 'fb1',
  'geotype': 'cd',
  'column_mapping': {'z': 'moe_foreign_born', 'p': 'pct_foreign_born'}}
mgraber commented 3 years ago

97 should resolve this.