damonmcc commented 1 year ago

The purpose of this issue is to help the team track the tasks that need to occur before kicking off a CPDB update.

Update source data

Read more about who to reach out to and how to update a specific dataset here

Received from agency partner and manually loaded into data libraries

[x] Capital Commitment Plan: fisa_capitalcommitments
[x] ddc_capitalprojects_infrastructure
[x] ddc_capitalprojects_publicbuildings
[x] dot_projects_bridges
[x] edc_capitalprojects - new data needs to be appended onto existing source dataset
[x] edc_capitalprojects_ferry - new data is often not received and this dataset is not updated with each release because nothing has changed.

Automatically loaded into data libraries

Before initiating a build of cpdb, verify the latest version in DigitalOcean matches the version in NYC OpenData

Projects

[x] cpdb_capital_spending - updated with action
[x] dot_projects_intersections https://data.cityofnewyork.us/Transportation/Street-and-Highway-Capital-Reconstruction-Projects/97nd-ff3i
[x] dot_projects_streets https://data.cityofnewyork.us/Transportation/Street-and-Highway-Capital-Reconstruction-Projects/jvk9-k4re
[x] dpr_capitalprojects https://www.nycgovparks.org/bigapps
[x] dpr_parksproperties https://data.cityofnewyork.us/Recreation/Parks-Properties/enfh-gkve
[x] dcp_cpdb_agencyverified (does not get updated)

Building and lot-level info

[x] dcp_mappluto_wi https://www1.nyc.gov/site/planning/data-maps/open-data/dwn-pluto-mappluto.page
[x] dcp_facilities https://www1.nyc.gov/site/planning/data-maps/open-data/dwn-selfac.page
[x] doitt_buildingfootprints https://data.cityofnewyork.us/Housing-Development/Building-Footprints/nqwf-w8eh

Spatial boundaries

Come from Geosupport update

[x] dcp_stateassemblydistricts https://www1.nyc.gov/assets/planning/download/pdf/data-maps/open-data/nyad_metadata.pdf?ver=21d
[x] dcp_ct2020 https://www1.nyc.gov/site/planning/data-maps/open-data/districts-download-metadata.page
[x] dcp_congressionaldistricts https://www1.nyc.gov/site/planning/data-maps/open-data/districts-download-metadata.page
[x] dcp_cdboundaries https://www1.nyc.gov/site/planning/data-maps/open-data/districts-download-metadata.page
[x] dcp_statesenatedistricts https://www1.nyc.gov/site/planning/data-maps/open-data/districts-download-metadata.page
[x] dcp_municipalcourtdistricts https://www1.nyc.gov/site/planning/data-maps/open-data/districts-download-metadata.page
[x] dcp_school_districts https://www1.nyc.gov/site/planning/data-maps/open-data/districts-download-metadata.page
[x] dcp_councildistricts https://www1.nyc.gov/assets/planning/download/pdf/data-maps/open-data/nycc_metadata.pdf

Other

[x] dcp_trafficanalysiszones - this dataset is almost never updated
[x] nypd_policeprecincts https://data.cityofnewyork.us/Public-Safety/Police-Precincts/78dh-3ptz
[x] fdny_firecompanies https://data.cityofnewyork.us/Public-Safety/Fire-Companies/iiv7-jaj9

damonmcc commented 1 year ago

this build version is dependent on #102

will send emails to request agency partner data ASAP

fvankrieken commented 1 year ago

Data requests have been made to from agency partners

Received:

[x] dot
[x] edc
[x] ddc
[x] fisa

fvankrieken commented 12 months ago

@AmandaDoyle - we currently version (in edm publishing) cpdb by date. I'd prefer to use the actual name of the release

Opendata has format of "23pre", "23exec", and "23adopt", so let's go with that?

fvankrieken commented 12 months ago

[ ] communicate with GIS about publishing, metadata, etc. Might need its own issue, but just wanted to log for now

damonmcc commented 11 months ago

@AmandaDoyle - we currently version (in edm publishing) cpdb by date. I'd prefer to use the actual name of the release

Opendata has format of "23pre", "23exec", and "23adopt", so let's go with that?

@fvankrieken not sure if you got a reply on this already. I like this idea, but maybe we should save that change for the first 2024 build for consistency in 2023?

damonmcc commented 11 months ago

revising data dictionary with Capital Planning and GIS to reflect recent changes to columns

damonmcc commented 10 months ago

working on distribution with GIS (issue in edm-overview repo here)

damonmcc commented 9 months ago

(I'm guessing here)

~the latest build of CPDB 23Q4 aka CPDB 23_adopt was from branch fvk-cpdb-fisa-enhancements and the S3 link to the zipped outputs is here~

fvankrieken commented 9 months ago

Oops sorry - that's the wrong output. Zipped outputs are here