NYCPlanning / db-developments

🏠 🏘️ 🏗️ Developments Database
https://nycplanning.github.io/db-developments
8 stars 2 forks source link

A2 made geocoding exorbitantly expensive #609

Closed td928 closed 1 year ago

td928 commented 1 year ago

After the enhancement of 22Q2 and A2 jobs added to the mix, it begins to seems that our data sync workflow would fail due to the sheer size of the number of records (more than 2 million with both bis and now).

I have some quick fixes implemented which seems to allow geocoding to finish but in the long run I do think it is worth to consider less the load and simplify the geocoding (see #597 #599) so we can both run devdb build more easily.