dataform-co / dataform

Dataform is a framework for managing SQL based data operations in BigQuery
https://cloud.google.com/dataform/docs
Apache License 2.0
852 stars 163 forks source link

Generated incremental SQL when table does not exist should not do a `create or replace` #1561

Open lewish opened 1 year ago

lewish commented 1 year ago

In the rare situation that seems to have happened to a customer recently - that BigQuery tells us a incremental table does not exist even though it does - we should not generate a create or replace statement to rebuild the table in case it does actually exist, and instead generate a create table if not exists statement in case.

lewish commented 1 year ago

Relevant code: https://github.com/dataform-co/dataform/blob/b5a561643e5020401639ef3a73aab6991083eb16/core/adapters/bigquery.ts#L34C46-L34C46

Should be: