adding manual load cpdb yml

NYCPlanning / db-data-library

📚 Data Library

https://nycplanning.github.io/db-data-library/library/index.html

MIT License

0 stars 1 forks source link

adding manual load cpdb yml #406

Open DeaBardhoshi opened 1 year ago

DeaBardhoshi commented 1 year ago

Added a manual loading YML for the CDPB historical projects + made a python script for merging different CPDB files together. Added NYCOC Checkbook YML that reads in checkbook data

athursland commented 1 year ago

submitted nycoc_checkbook.yml in my PR so we don't need the duplicate - also, I think you need to specify the source path on your local machine (it's referencing my folder directories right now) path: /Users/alexandrathursland/Documents/NYC-DCP/historical-spend-data/checkbook_citywide_agencies.csv

athursland commented 1 year ago

regarding library/script/dcp_cpdb.py:

you're reading the cpdb geometries data in as pandas dataframes, combining and then writing to a CSV, but the cpdb geometries are shape files not csvs - and we originally used GeoPandas rather than pandas to concatenate them. I'm not 100% sure but this may cause an issue with interacting with a GeoPandas df down the line
- cpdb_projects.csv doesn't need to be joined onto the cpdb geometries (cpdb geometries contains all the same info plus geometries)