NYCPlanning / data-engineering

Primary repository for NYC DCP's Data Engineering team

20 stars 0 forks source link

CPDB 24exec #860

Closed damonmcc closed 1 month ago

damonmcc commented 4 months ago

Product Name

cpdb

Build Version

24exec

Status of Update

[x] source data extracted
[x] draft build succeeded
[x] draft build passed QA
[x] data packaged and distributed

damonmcc commented 4 months ago

@fvankrieken started requesting source data from agencies via email

fvankrieken commented 2 months ago

Update source data

Read more about who to reach out to and how to update a specific dataset here

Received from agency partner and manually loaded into data libraries

[x] Capital Commitment Plan: fisa_capitalcommitments
[x] ddc_capitalprojects_infrastructure
[x] ddc_capitalprojects_publicbuildings
[x] dot_projects_bridges
[x] edc_capitalprojects - new data needs to be appended onto existing source dataset
[x] edc_capitalprojects_ferry - new data is often not received and this dataset is not updated with each release because nothing has changed.

Automatically loaded into data libraries

Before initiating a build of cpdb, verify the latest version in DigitalOcean matches the version in NYC OpenData

Projects

[x] nycoc_checkbook - updated with action
[x] dot_projects_intersections https://data.cityofnewyork.us/Transportation/Street-and-Highway-Capital-Reconstruction-Projects/97nd-ff3i
[x] dot_projects_streets https://data.cityofnewyork.us/Transportation/Street-and-Highway-Capital-Reconstruction-Projects/jvk9-k4re
[x] dpr_capitalprojects https://www.nycgovparks.org/bigapps
[x] dpr_parksproperties https://data.cityofnewyork.us/Recreation/Parks-Properties/enfh-gkve
[x] dcp_cpdb_agencyverified (does not get updated)

Building and lot-level info

[x] dcp_mappluto_wi https://www1.nyc.gov/site/planning/data-maps/open-data/dwn-pluto-mappluto.page
[x] dcp_facilities https://www1.nyc.gov/site/planning/data-maps/open-data/dwn-selfac.page
[x] doitt_buildingfootprints https://data.cityofnewyork.us/Housing-Development/Building-Footprints/nqwf-w8eh

Spatial boundaries

Come from Geosupport update

[x] dcp_stateassemblydistricts https://www1.nyc.gov/assets/planning/download/pdf/data-maps/open-data/nyad_metadata.pdf?ver=21d
[x] dcp_ct2020 https://www1.nyc.gov/site/planning/data-maps/open-data/districts-download-metadata.page
[x] dcp_congressionaldistricts https://www1.nyc.gov/site/planning/data-maps/open-data/districts-download-metadata.page
[x] dcp_cdboundaries https://www1.nyc.gov/site/planning/data-maps/open-data/districts-download-metadata.page
[x] dcp_statesenatedistricts https://www1.nyc.gov/site/planning/data-maps/open-data/districts-download-metadata.page
[x] dcp_municipalcourtdistricts https://www1.nyc.gov/site/planning/data-maps/open-data/districts-download-metadata.page
[x] dcp_school_districts https://www1.nyc.gov/site/planning/data-maps/open-data/districts-download-metadata.page
[x] dcp_councildistricts https://www1.nyc.gov/assets/planning/download/pdf/data-maps/open-data/nycc_metadata.pdf

Other

[x] dcp_trafficanalysiszones - this dataset is almost never updated
[x] nypd_policeprecincts https://data.cityofnewyork.us/Public-Safety/Police-Precincts/78dh-3ptz
[x] fdny_firecompanies https://data.cityofnewyork.us/Public-Safety/Fire-Companies/iiv7-jaj9

damonmcc commented 2 months ago

built and sent to Capital Planning team

fvankrieken commented 1 month ago

Passed QA w/ capital planning at end of last week, notified @caseysmithpgh today!

caseysmithpgh commented 1 month ago

CPDB local distribution went swimmingly today. Will send to webteam with the rest of Julys data, likely later this week.