dbt-labs / dbt-external-tables

dbt macros to stage external sources
https://hub.getdbt.com/dbt-labs/dbt_external_tables/latest/
Apache License 2.0
297 stars 119 forks source link

Add self reference for macro #150

Closed lassebenni closed 1 year ago

lassebenni commented 2 years ago

Explicitly reference the helper macro 'spark__recover_partitions'

Otherwise running stage_external_sources in a Model will error with 'spark__recover_partitions' is undefined.

Description & motivation

When trying to run stage_external_sources in a Model e.g.

my_model.sql

{{
    config(
         pre_hook=[
            "{{ dbt_external_tables.stage_external_sources( select='default.my_external_table' ) }}",
        ]
    )
}}

dbt errors with

17:58:31    'spark__recover_partitions' is undefined
17:58:31    
17:58:31    > in macro get_external_build_plan (macros/common/get_external_build_plan.sql)
17:58:31    > called by macro stage_external_sources (macros/common/stage_external_sources.sql)
17:58:31    > called by macro spark__get_external_build_plan (macros/plugins/spark/get_external_build_plan.sql)

This is due to the helper macrospark__recover_partitions not being self-referenced: dbt_external_tables.spark__recover_partitions. This commit fixes this.

Checklist

github-actions[bot] commented 1 year ago

This PR has been marked as Stale because it has been open with no activity as of late. If you would like the PR to remain open, please comment on the PR or else it will be closed in 7 days.

github-actions[bot] commented 1 year ago

Although we are closing this PR as stale, it can still be reopened to continue development. Just add a comment to notify the maintainers.