search

NYCPlanning / db-developments

🏠 🏘️ 🏗️ Developments Database
https://nycplanning.github.io/db-developments
8 stars 2 forks source link

ingest manual corrections 22Q2 #510

Closed td928 closed 2 years ago

td928 commented 2 years ago

Overview

one reviewer is enough for this small work. 👶

Pulling the spreadsheets sent by Sam in emails and replaces the spreadsheet in the data folder. And also change the version number to 22Q2 according to the notes in this issue.

QAQC

I also looked at the corrections_applied and corrections_not_applied. Only 14 corrections from the 2022 July round of research, and most of them "make sense" in that they are not applied for the simple reason the pre corrected value does not match with the manual input old values. I checked the hny_no_match table to see if any of the manually added records from erroneously present in there and found no such cases.

Review PR

Some caveat about rerun the corrections table. Since the corrections are applied throughout the build process, the entire 02_build_devdb should be rerun and also along with the 01_dataloading since the correction table is loaded at the very end the dataloading steps.

mbh329 commented 2 years ago

lgtm. Tested this by running the 01_dataloading.sh and 02_build_devdb. Looks like the corrections were applied successfully