21Q4 UPDATE

[AmandaDoyle]

Update code

• [ ] Add most recent full/half year to aggregate tables

Update source data

• [x] Update version.env to:

  CAPTURE_DATE=2022-01-16
  CAPTURE_DATE_PREV=2021-01-01
  DOB_DATA_DATE=2022/01/01
  VERSION=21Q4
  VERSION_PREV=20Q4

  Make sure the following are up-to-date in recipes:

• [x] dcp_mappluto
• [x] dof_shoreline updated with zoningtaxlots, safe to ignore
• [x] council_members check opendate
• [x] doitt_buildingfootprints check opendata
• [x] doitt_buildingfootprints_historicalcheck opendata
• [x] doitt_zipcodeboundaries -> never changed, safe to ignore
• [x] doe_school_subdistricts -> received from capital planning
• [x] doe_eszones -> the url for this changes year by year, search on opendata
• [x] doe_mszones -> same as above
• [x] hpd_hny_units_by_building check opendata

DCP Admin Boundaries from Bytes

• [x] dcp_cdboundaries
• [x] dcp_censusblocks
• [x] dcp_censustracts
• [x] dcp_school_districts
• [x] dcp_boroboundaries_wi
• [x] dcp_councildistricts
• [x] dcp_firecompanies
• [x] dcp_policeprecincts

DOB data

• [x] dob_cofos -> manually updated, received by email
• [x] dob_jobapplications check actions
• [x] dob_permitissuance check actions
• [x] dob_now_applications -> manually updated, received by email

[mbh329]

dcp_mappluto - updated 20 days ago (latest and 21v4) dof_shoreline - updated 8 days ago (latest and 20220110 council_members - updated 9 months ago in DO (most recent folder doesn't align with dates), updated 20210630 in OpenData. This dataset should be updated on opendata to reflect most recent council member changes doitt_buildingfootprints - updated 3 days ago (latest and 20220116) doitt_buildingfootprints_historical - updated one year ago in DO (latest and 20210216), open data says this was last updated on today on 20220118 doitt_zipcodeboundaries - last updated 9 months ago (latest and 20210405) doe_school_subdistricts - latest folder last updated 8 months ago, version folder is out of date - uses a 2017 folder to save the most recent data (should update this to reflect the actual updated date or version) doe_eszones - last updated 9 months ago that reflect the 2019-2020 data from. https://data.cityofnewyork.us/Education/2019-2020-School-Zones-Elementary-/kuk3-ypca doe_mszones - last updated 9 months ago in DO that reflect the 2019-2020 data from https://data.cityofnewyork.us/Education/2019-2020-School-Zones-Middle-School-/ie4z-reuy

[SashaWeinstein]

Updating the version.env and the admin boundaries was mostly smooth with the one issue that dcp_policeprecincts was misspelled in the quarterly updates workflow in data library. This was fixed and now all admin boundaries are automatically refreshed in that workflow

[AmandaDoyle]

Regarding the add most recent full/half year to aggregate tables step. Update yearly.sql and aggregate.sql to include comp2021 field

[SashaWeinstein]

Te is taking a look at this, in the meantime I want to flag that there is no aggregate.sql in the main branch of devDB

[td928]

Hey @AmandaDoyle I want to follow up on your comment about the aggregate tables step. I think @SashaWeinstein is right that the yearly.sql and aggregate.sql have been refactored to multiple new aggregate queries and they should handle the 2021 just fine as they are. Although it currently is producing only for 2010 census geometries, I am taking a look on adding the 2020 census geometries.