NYCPlanning / db-developments

🏠 🏘️ 🏗️ Developments Database
https://nycplanning.github.io/db-developments
8 stars 2 forks source link

doc # == '01' is no longer filtering BIS down to unique job number #502

Closed td928 closed 2 years ago

td928 commented 2 years ago

As Jul 1 2022 build, it is spotted in the _init_bis_devdb step that the records by job number is no longer unique after filtering down the doc # ==1. This cause a series of downstream issue:

  1. the unique id create from ogr_fid as uid is no longer true unique identifier in the database with many duplicates
  2. in the table joining operations where the assumption is the records are unique and this leads to tables become too large for build