konigio / konig

A collection of tools for semantic modeling
4 stars 4 forks source link

Abandon Overlay ETL if record matches current state #1637

Open gmcfall opened 5 years ago

gmcfall commented 5 years ago

A new record should be added to the BigQuery target table only if it is a new record.

Acceptance Criteria

In the overlay pattern, we use the most recent row from the target table as a source and "overlay" the properties with properties from the source records. However, this process should result in a new row in the target table only if it is different from the most recent record.