NYCPlanning / db-developments

🏠 🏘️ 🏗️ Developments Database
https://nycplanning.github.io/db-developments
8 stars 2 forks source link

add associative match query #505

Closed td928 closed 2 years ago

td928 commented 2 years ago

For more details on issue see #504

Solution

Since the issue was with the matches from are not effectively grouping together and causing failed joins down the line due to logical contradiction. I figure the easies solution would be to make sure the grouping between all methods and add the missing matches to the HNY_matches. Not sure my query is the most elegant in terms of creating the associative table but it did seems to work now the build step can run to end without issue and does seems the HNY_matches table is updated with all the hny_id and job_number matches.

Next step

Once this is merged into the feature branch we can turn our attention to merge in the #501. And we should be almost ready for a production run. Thanks!

SashaWeinstein commented 2 years ago

Digging into this now, my first question is what "associative" means in this context?

td928 commented 2 years ago

Digging into this now, my first question is what "associative" means in this context?

good questions. Kind of thinking about the mathematical associative entity e.g. a=b b=c => a=c. In our case, if hny_id is associated with one job number and another hny_id and the other hny_id is therefore should also associate with the job number by definition.

mbh329 commented 2 years ago

@td928 I am getting the same error as before in the build step

td928 commented 2 years ago

@SashaWeinstein the deduplication with the hny_id||job_number did not work unfortunately with deduplicating but the a.job_number||b.job_number did work! Let me know if this approach makes sense

SashaWeinstein commented 2 years ago

After talking this through, the deduping on j1 and j2 does make sense