search

NYCPlanning / data-engineering

Primary repository for NYC DCP's Data Engineering team
14 stars 0 forks source link

fix missing GFT lots in exported source data #823

Closed damonmcc closed 2 weeks ago

damonmcc commented 2 weeks ago

resolves #815

successful build here. all builds on this branch here

I found which models to change by searching for all models in /product/sources that select records WHERE ST_GEOMETRYTYPE(raw_geom) = 'ST_MultiPolygon' and changing the model they select from.

See last commit message for explanation of the substantial changes to int_buffers__lpc_landmarks. GIS is ok with (and actually likes) not seeing source points when lots where they were joined to lots.

screenshots

before

Screenshot 2024-05-01 at 12 39 44 PM

after

Screenshot 2024-05-01 at 12 40 10 PM
damonmcc commented 2 weeks ago

@fvankrieken

Is it potentially worth using ST_MULTI on just about every data source, just to make sure we don't have other

good call, but I didn't wanna go too far since (especially historic) source data seems to be in flux at the moment and we wanna make a pass at standardizing stuff more generally

all good if we roll with this while Jack and I keep an eye out for other issues?