search

NYCPlanning / data-engineering

Primary repository for NYC DCP's Data Engineering team
14 stars 0 forks source link

FacDB - fix issues in 24v1 #804

Closed sf-dcp closed 2 weeks ago

sf-dcp commented 3 weeks ago

Related to #461.

What

During the 24v1 review, GIS identified this issue:

Why

When creating the qc_diff table used in the QA review, we filter out records with empty values in the geom column. It appears that dfta_contracts records weren't getting geocoded by geosupport in the beginning of the build. Specifically, the source zip code column name changed in this data version, and our code silently failed, returning empty values from geosupport

Solution: fix zip code column name

Build here.

As can be seen, now dfta_contracts records are getting geocoded: count_new is not zero. image

Old records for comparison:

image
sf-dcp commented 3 weeks ago

nice, did u wanna build on main with the pinned versions? or gonna drop the commit now that the build on this branch succeeded?

I was going to drop the commit before merging, unless you suggest otherwise?