This fixes #13 in a sort of funky way: by parsing the SQL files for dataset transformations via sqlparse and using a bit of static analysis to figure out what tables are being created.
Another approach would be to simply list all the tables in the temporary schema after executing the transformations, but there were a number of places get_dataset_tables were being used, and some of them might not have assumed that they have a live postgres instance to query. Also, I always wanted to play around with SQL parsing.
Anyways, if this ends up not working out in the long run we can always revert it.
This fixes #13 in a sort of funky way: by parsing the SQL files for dataset transformations via
sqlparseand using a bit of static analysis to figure out what tables are being created.Another approach would be to simply list all the tables in the temporary schema after executing the transformations, but there were a number of places
get_dataset_tableswere being used, and some of them might not have assumed that they have a live postgres instance to query. Also, I always wanted to play around with SQL parsing.Anyways, if this ends up not working out in the long run we can always revert it.
This also updates our version of NYCDB to https://github.com/aepyornis/nyc-db/commit/6e95f7884578670dac8207819ef37d217dbbd540.