move-coop / parsons

A python library of connectors for the progressive community.
https://www.parsonsproject.org/
Other
257 stars 125 forks source link

[Feature/Addition] Add support for BigQuery export/extract to Google Cloud Storage #921

Open austinweisgrau opened 8 months ago

austinweisgrau commented 8 months ago

BigQuery can export a query result to a file in Google Cloud Storage. This is helpful for various workflows, e.g. running a query with a huge result. Having this feature will also lead to closer parity with the Redshift connector and its "unload" method.

Possible Implementation

Here's the documentation on this including an example implementation in python: https://cloud.google.com/bigquery/docs/exporting-data#export-data-in-bigquery

Priority

medium