datamade / just-spaces

🏕 A tool from University City District and DataMade to promote better and more just public spaces
https://justspacesproject.org
MIT License
7 stars 1 forks source link

Make ACS data imports resistant to updates #158

Closed jeancochrane closed 5 years ago

jeancochrane commented 5 years ago

Right now, the import_data command truncates the CensusObservation table and bulk-imports the CSV output from the Makefile. We should modify this to make it iterate the CSV, check if a CensusObservation exists matching the FIPS code and variable name, and optionally update that record.

It would also be useful to add a --truncate flag to the command to allow us to truncate the table and start over if it gets into a weird state.