NYCPlanning / db-developments

🏠 🏘️ 🏗️ Developments Database
https://nycplanning.github.io/db-developments
8 stars 2 forks source link

22Q2 censusunits10 is wrong in the aggregate table #543

Open td928 opened 2 years ago

td928 commented 2 years ago

housing team found that the value for censusunits10 is off in the final aggregate table.

td928 commented 1 year ago

to connect the loose ends on this issue. While working on publishing enhancement some works were done to the aggregate logics and new census units logics should built on the top of this work.

td928 commented 1 year ago

Population shared with us this dataset for census housing units count 2020 2020 housing units by block.xlsx

Question is we decided awhile ago that we are no longer gonna rely on the census block to roll up to the other geographies because there are some edge cases where it does not break up nicely. So we are going to request units count for the all geographies we are going to produce aggregate files on from Population directly.

td928 commented 1 year ago

Response from Population about new data going to be delivered to DE:

This will be doable for each of the geographies you have listed except for City Council Districts. I will be out again tomorrow and I do have competing work program demands, but I expect to be able to deliver the files to you early next week. Also just want to note, in order to publish official statistics for Community Districts, Population Division had to allocate some Census Blocks that straddled boundaries to one CD or another based on where the majority of housing units were. For all the other geographies on your list, 2020 Census Blocks nest neatly within and no such allocation was needed.

This is inline what we expected and for City Council Districts we got greenlights from Housing to drop them from the aggregate table which means the census units won't be included for CCD file.

One main question remains for Housing to decide is whether the census units 2010 is still relevant and as a consequence since_cen10 are probably eliminated as well.

I have some mock up work on this feature branch for this work but only has the census block 2020 data. It could be expanded once the data was delivered by Population.

AmandaDoyle commented 1 year ago

@td928 Adding some detail here so that this is fully scoped out. Please add / edit as needed

In the aggregation tables there is a column capturing the census unit count for the relevant decade (i.e. 2010 or 2020 census_units20) . We need to add this column to the 2020 aggregate tables by: 1) Receiving the data for each geography from the Population team 2) Loading file(s) / new data tables from pop into data library 3) Updating SQL scripts to incorporate new field into output tables. Work in progress with block level data is here.