NYCPlanning / ceqr-app-data-archive

(DEPRECATED)data pipelines for CEQR app, managed by data engineering
https://github.com/NYCPlanning/ceqr-app-data
1 stars 1 forks source link

Mismatch in the number of DEP CATS permits #47

Closed CliangDCP closed 4 years ago

CliangDCP commented 4 years ago

Please investigate the number of DEP CATS permits comparing to what EARD has purged with the following categories:

CATS Permits raw 010720.zip

baolingz commented 4 years ago

@CliangDCP Thanks for pin us on this. We've already figured out what made the discrepancies happen for the DEP CATS permits dataset and fixed the ETL scripts but we are still trying to improve its geocoding performance by addressing issues 45 . We will ship you the latest version by the end of the week. Feel free to pin us if you have any other questions.

CliangDCP commented 4 years ago

Got it, thank you!

baolingz commented 4 years ago
SELECT COUNT(*) FROM dep_cats_permits.latest
WHERE TRIM(status) != 'CANCELLED'
AND LEFT(applicationid, 1) != 'G'
AND (LEFT(applicationid, 1) != 'C' OR (requesttype != 'REGISTRATION' AND requesttype != 'REGISTRATION INSPECTION' AND requesttype != 'BOILER REGISTRATION II'))
AND (LEFT(applicationid, 2) != 'CA' OR requesttype != 'WORK PERMIT' OR TRIM(status) != 'EXPIRED')
;