ITD data flow improvements

[damonmcc]

Project Description

Since DE is upstream of other teams in ITD and maintains some of the relevant automations, we will improve and standardize how data flows to those teams for QA and distribution.

TImeline

Week of 2/16

• [x] DE meets with AE to share how we use github actions for data flow
• [x] https://github.com/NYCPlanning/data-engineering/issues/649

Future

• [x] #641
• [ ] #564
• [ ] #227

---

Ideas from 2/1 meeting

• GDE <> SPM
  • avoid duplicate issue creation in edm-data-operations
• DE <> GIS
  • when ready for QA, promote a build and create a GH issue for GIS
  • use a staging directory for data to QA
• DE <> AE
  • how to update data in Zoning API backend DB
  • how to communicate/validate AE’s data expectations

[damonmcc]

whiteboard from 2/1 meeting, we focused on GIS team