NYCPlanning / data-engineering

Primary repository for NYC DCP's Data Engineering team
14 stars 0 forks source link

revise GFT lot zoning logic #789

Closed damonmcc closed 3 weeks ago

damonmcc commented 3 weeks ago

resolves #741

successful build here, all build runs here

changes

The most significant change is to the model int_flags__zoning which now uses the new model int__zoning_districts. Since the diff is hard to read, here are links to the changed models: int_flags__zoning, int__zoning_districts.

Per chats with GIS, this also changes the final columns related to zoning districts to be most useful.

notes

Chose to put the new model in the /intermediate/flags folder to avoid creating a new yml file, but understand if it'd make more sense to move it up to `/intermediate'.

Notes on column changes in the final table:

DAG screenshot

Screenshot 2024-04-24 at 2 52 58 PM

damonmcc commented 3 weeks ago

for readability and our future selves, gonna pull out the zoning district logic from int_flags__zoning into a new model that feeds into it

damonmcc commented 3 weeks ago

@fvankrieken I changed the group/order bys to use column names rather than numbers and moved the last group by in int_flags__zoning to the last CTE

appreciate your suggestions for changing part of int__zoning_districts to use your preferred approaches, but it isn't worth holding up this PR which also fixes GFT being broken on main. let's save it for a follow-up