search

NYCPlanning / ae-data-flow

Data pipelines to populate Application Engineering databases
1 stars 0 forks source link

update digram for Zoning API data #15

Closed damonmcc closed 3 months ago

damonmcc commented 3 months ago

related to https://github.com/NYCPlanning/data-engineering/issues/660

One of the more significant changes reflects our plan to use a distinct database cluster for data flow (rather than using the same API's DB).

This design choice is meant to avoid "littering" the API DB with source tables and to not dedicate any API DB bandwidth to "data flow" operations.

The last step name "replace API tables" is the only operation we'll perform on the API DB.

damonmcc commented 3 months ago

gonna merge this that it's easy to reference elsewhere (the project issue especially)

but of course always down to make edits!