search

NYCPlanning / db-cpdb

Capital Projects Database
https://nycplanning.github.io/db-cpdb
0 stars 0 forks source link

Maintenance, multiple minor issues #134

Closed fvankrieken closed 1 year ago

fvankrieken commented 1 year ago

So far changes include

Closes #113 - remove call to frame.append which is deprecated (method to append rows to data frame). Now uses pd.concat instead

Closes #115 - pretty self-explanatory. SQL scripts that began DROP VIEW ... IF EXISTS are changed to DROP TABLE ... IF EXISTS instead

Closes #133 - self-explanatory

fvankrieken commented 1 year ago

This commit addresses an issue I had both locally and saw in the github action. See below article for explanation.

https://levelup.gitconnected.com/how-to-fix-attributeerror-optionengine-object-has-no-attribute-execute-in-pandas-eb635fbb89e4

fvankrieken commented 1 year ago

I'm guessing screenshots are not are prefered method of testing verification but for now that was simplest for showing behavior for dot_bridges.py

image

fvankrieken commented 1 year ago

So I think there might be at least one issue in the build that could have affected the output data for the latest build of cpdb

See here

This is the problem addressed by my last comment. It results in the table dcp_cpdb_agencyverified_geo not being created, which contains geo data for 700+ maprojids

Once I clean stuff up a bit, I could run a build from this branch and do a little bit of comparing, see if there's anything drastically different from the last build from main

fvankrieken commented 1 year ago

CI build here

fvankrieken commented 1 year ago

Build went smoothly, output data looks good by my QCing (no visible differences in QCQC app from main, so seems that maybe above mentioned issue did not affect the build a few days ago)