databricks / dbt-databricks

A dbt adapter for Databricks.
https://databricks.com
Apache License 2.0
226 stars 119 forks source link

1.9 renamed internal aliases `DBT_INTERNAL_DEST / DBT_INTERNAL_SOURCE` to `tgt / src` which is causing errors in some runs #841

Closed jeremyyeo closed 3 weeks ago

jeremyyeo commented 3 weeks ago

Describe the bug

We renamed "well-known" internal aliases DBT_INTERNAL_X here: https://github.com/databricks/dbt-databricks/blob/fec8b30985a2f7007c8ff04866623d7a37238236/dbt/include/databricks/macros/materializations/incremental/strategies.sql#L75-L76

Some users have macros that work off the assumption that those internal aliases would be kept to DBT_INTERNAL_X - this generates an invalid SQL like:

    merge
    into
        `db`.`schema`.`tbl` as tgt
    using
        `tbl__dbt_tmp` as src
    on

            DBT_INTERNAL_DEST.updated_at >= '2024-10-29'
            AND DBT_INTERNAL_DEST.updated_at <= '2024-10-29'

    and 
                  src.updated_at <=> tgt.updated_at

    and 
                  src.id <=> tgt.id

...

    when matched
...

as DBT_INTERNAL_X alias does not exist.

Steps To Reproduce

In as much detail as possible, please provide steps to reproduce the issue. Sample data that triggers the issue, example model code, etc is all very helpful here.

Expected behavior

Worth renaming it back to DBT_INTERNAL_X.

Screenshots and log output

If applicable, add screenshots or log output to help explain your problem.

System information

The output of dbt --version:

<output goes here>

The operating system you're using:

The output of python --version:

Additional context

Add any other context about the problem here.