databricks / dbt-databricks

A dbt adapter for Databricks.
https://databricks.com
Apache License 2.0
225 stars 119 forks source link

Support for `MERGE WITH SCHEMA EVOLUTION` #707

Open stevenayers-bge opened 4 months ago

stevenayers-bge commented 4 months ago

Describe the feature

Add support for the new MERGE WITH SCHEMA EVOLUTION syntax available on the current and preview channel SQL Warehouse versions. https://docs.databricks.com/en/delta/update-schema.html#schema-evolution-syntax-for-sql.

This is especially useful if you need schema evolution on struct data types.

The change is quite easy SQL wise, but I don't know how you would want to expose the option... via on_schema_change?

mi-volodin commented 3 months ago

@stevenayers-bge Hi, just to be sure I understand correctly. What is the difference between running this and setting the parameter for the table?

Based on Databricks documentation there are no differences.

Some use case might be helpful.

mi-volodin commented 2 months ago

Fixed in #739