dbt-labs / dbt-spark

dbt-spark contains all of the code enabling dbt to work with Apache Spark and Databricks
https://getdbt.com
Apache License 2.0
395 stars 221 forks source link

dbt_clone macros, materialization and tests to dbt-spark #816

Closed McKnight-42 closed 1 year ago

McKnight-42 commented 1 year ago

resolves #815

Description

adding the macros, and customer materialization needed on adapter side for dbt_clone and trying to design adapter zone version of Clone tests from defer_state in https://github.com/dbt-labs/dbt-core/pull/7881

Checklist

McKnight-42 commented 1 year ago

testing update 6/29: passing test_dbt_clone.py passing locally. failing in ci/cd

McKnight-42 commented 1 year ago

@jtcohen6 Curious I know we want the shallow clone Do we also want to do deep copy as an option? I noticed it is not mentioned in updated comments on https://github.com/dbt-labs/dbt-core/issues/7549 but still in the original draft pr https://github.com/dbt-labs/dbt-core/pull/7258.

jtcohen6 commented 1 year ago

@McKnight-42 IMO no need to support deep copy/clone as an option! As I understand it, this is the same as fully recreating the table (create table as select * from {other}), and loses a lot of the cost/speed/performance benefits of using clone.