search

kevinburke / gobike

https://bikeshare.science
6 stars 0 forks source link

Generate datasets for import #4

Closed kyleconroy closed 6 years ago

kyleconroy commented 6 years ago

Remove the dbload command and replace it with gobike-dataset, which generates a single CSV suitable for import into Postgres or BigQuery. I've also moved all the geo-related functionality into a separate package.

kyleconroy commented 6 years ago

I've confirmed that the generated CSV can be imported into BigQuery. Also, it takes about 30 seconds to import the data into Postgres via COPY FROM, which is way faster than dbload ever was.