ccao-data / data-architecture

Codebase for CCAO data infrastructure construction and management
https://ccao-data.github.io/data-architecture/
5 stars 3 forks source link

Add fed and state congressional district nums #469

Closed wrridgeway closed 3 weeks ago

wrridgeway commented 1 month ago

Just a quick addition of some columns in vw_pin_universe that are functions of pre-existing columns.

wrridgeway commented 1 month ago
select distinct
    census_data_year,
    census_congressional_district_geoid,
    census_congressional_district_num,
    census_state_representative_geoid,
    census_state_representative_num,
    census_state_senate_geoid,
    census_state_senate_num,
    census_acs5_data_year,
    census_acs5_state_representative_geoid,
    census_acs5_state_representative_num,
    census_acs5_state_senate_geoid,
    census_acs5_state_senate_num
from z_dev_wridgeway_default.vw_pin_universe

output

wrridgeway commented 1 month ago

issue (blocking): What's up with the 2011 IDs in the output? We may want to fix those in the source.

yeah, not sure. i'll take a look. i assumed that it was an expected change in formatting.

wrridgeway commented 1 month ago
select distinct
    census_data_year,
    census_congressional_district_geoid,
    census_congressional_district_num,
    census_state_representative_geoid,
    census_state_representative_num,
    census_state_senate_geoid,
    census_state_senate_num,
    census_acs5_data_year,
    census_acs5_state_representative_geoid,
    census_acs5_state_representative_num,
    census_acs5_state_senate_geoid,
    census_acs5_state_senate_num
from z_dev_wridgeway_default.vw_pin_universe

new output

We'll need to trigger a rebuild of location.census after this is merged.

wrridgeway commented 1 month ago

All gone.

wrridgeway commented 1 month ago

Did a quick investigation into which parcels are missing census geoids for 2024:

image

dfsnow commented 4 weeks ago

Did a quick investigation into which parcels are missing census geoids for 2024:

  • 1026 pin10s are missing lat/lon so can't be spatially joined to census
  • 22 pin10s are in the lake, here's a sample:

image

@wrridgeway I'm guessing this is also the cause of other missing census geographies? (e.g. null congressional district in 2022 sometimes, even though other census geos are filled?)

wrridgeway commented 3 weeks ago

I double checked this data, looking at 2016 which has the most null geoids (~12,000), and the vast majority is due to NA lat/long. Only about 800 are missing geoids from the spatial join, and those are from the same lake coast issue.