Closed butanian closed 6 years ago
cc @sanjaybv
I definitely want to combine logic between the redshift and bigquery implementations. Maybe even make a generic way that can be applied to other potential new implementations! There are some subtle differences between the two, you pointed one out, and the other is the Schema objects the two implementations use are different. I was hoping to discuss it some and then potentially do it in a different PR since it doesn't affect behavior and you know, how big a fan I am of granular PRs 😬
This PR adds BigQuery support for schema changes. If hauser fields are not present in the existing export table in the specified dataset, we now add them and make sure all hauser fields are also present on any existing dataset table.
This is part (2) of a larger arc of work which will - (1) Add support for changing export schema on Redshift (2) Add support for changing export schema on BigQuery (3) Add new fields to the export