search

dbt-labs / dbt-external-tables

dbt macros to stage external sources
https://hub.getdbt.com/dbt-labs/dbt_external_tables/latest/
Apache License 2.0
297 stars 119 forks source link

Snowflake delta support #138

Closed aPeterHeise closed 1 year ago

aPeterHeise commented 2 years ago

Description & motivation

Added support for delta lake as described in #137

aPeterHeise commented 2 years ago

See here for feature description.

aPeterHeise commented 2 years ago

Hi @jtcohen6 , how can I convince you to accept this PR? ;-)

sdrzymala commented 2 years ago

Hi @jtcohen6 , There is more people and teams actually waiting for that PR. Any particular reason why it shouldn't be closed? Anyhow thanks a lot for your time! And big kudos to @aPeterHeise !

pgoslatara commented 1 year ago

@jtcohen6 Can this PR get some love please? Currently using this fix at a client so would be great to get it onto main.

jeremyyeo commented 1 year ago

Hey @aPeterHeise, thanks for the contribution - sorry for the delay here.

Just looking over this and looks straight forward and the docs here seem to suggest that AUTO_REFRESH needs to be FALSE when table format is delta?

Jerco tested this out and found that indeed that Snowflake does raise an error if we have a wrong combination:

"SQL compilation error: invalid property combination 'AUTO_REFRESH'='true' and 'TABLE_FORMAT'='delta'"

We don't have a way to test this in CI right now but this change looks good.