Fix bug on convert lines to polygons step

[SashaWeinstein]

Fix convert lines to polygons

In previous runs of the build workflow we were getting this error psql:sql/attributes_geomclean.sql:4: ERROR: GEOSBuffer: TopologyException: No forward edges found in buffer subgraph Googling the error led us to this blog post which suggested implementing the PostGIS ST_SnapToGrid function. This did fix the issue, you can see on runs from the 109 branch that this error no longer occurs.

Minor Upgrades

I made some other small upgrades while working with the build attribute step. 1) Fixed spelling error in the name of the step in github actions 2) Replaced frame.append() with pd.concat in attributes_maprojid_parkid 3) Updated attributes_maprojid_parkid to use dotenv python package 4) Added -q flag to suppress INSERT 0 1 messages that were filling up the logs in the github actions. It's much easier to navigate the logs now

[mbh329]

@SashaWeinstein this is my fault but I left in the two echo statements in the config.sh file that throw the 404 error, do you want me to pull down the branch and remove these + commit the changes?

[SashaWeinstein]

That would be great Max! Good catch

[td928]

@SashaWeinstein not sure if this was on your radar in the run you posted around line 29 in the third step there is error which I am also getting when building locally as well

psql:sql/attributes_maprojid_cd.sql:15: ERROR: GEOSIntersects: TopologyException: side location conflict at -73.882000000000005 40.866999999999997

[mbh329]

Before I look through the output, I took a look at the logs in the github actions. Error regarding the postgis function seems to be resolved with @td928 solution which is awesome! Two things to note: the curl could not resolve host "error/issue" still persists and (a minor) improvement we could make is updating the dot_bridges python file with concat to clean up the logs a bit more. Not sure if the curl could not resolve host is actually an issue at this point

[mbh329]

@AmandaDoyle @td928 @SashaWeinstein Are you familiar with theNOTICE: Self-intersection at or near point -73.997 40.572 message that is appearing in the build step? The second post in this blog seems to be helpful - https://gis.stackexchange.com/questions/157091/cleaning-geometries-in-postgis. can we chat about it at stand up?

[mbh329]

I ran this branch on my local machine and everything looked the same as the github actions build (in terms of log messages) until I got to the analysis part and the projects_by_community_dist_spending_Dateand projects_by_communitydist_spending are not generated locally - did anyone else have this issue?

[td928]

Before I look through the output, I took a look at the logs in the github actions. Error regarding the postgis function seems to be resolved with @td928 solution which is awesome! Two things to note: the curl could not resolve host "error/issue" still persists and (a minor) improvement we could make is updating the dot_bridges python file with concat to clean up the logs a bit more. Not sure if the curl could not resolve host is actually an issue at this point

about the dot_bridges I want to confirm with @AmandaDoyle. It seems to me possible this work was written when dot_bridges data was provided to use in a different format. It seems to me that new dataset is given to us in the same format as the transformed dot_projects_bridges_byfms if it is indeed the case we could remove the dot_bridges.py entirely

[mbh329]

Couple outstanding question for @AmandaDoyle:

• Where does the 11_spending.sh script fit into the local build? It seems that there are one table (cpdb_capital_spending) generated in this shell script but unsure as to where this fits in/needs to be run in order for the local build to properly work
• Is the capital_spending table being generate for anyone?

[SashaWeinstein]

Addressed issue #108