dbt-labs / dbt-external-tables

dbt macros to stage external sources
https://hub.getdbt.com/dbt-labs/dbt_external_tables/latest/
Apache License 2.0
315 stars 123 forks source link

stage all external sources related to a source_name broken #327

Open bullamiders opened 5 days ago

bullamiders commented 5 days ago

Describe the bug

In Version 0.10.0 it is not possible anymore to stage all external sources related to a source_name.

In my understanding:

  1. node.name references a table in a source while node.source_name references the whole source
  2. a single table of a source is handled here
  3. a whole source is handled here

The change introduced in the issue #309 and the related commit are probably the source, because the code part that handles a whole source is restricted to a single table now.

Steps to reproduce

# stage all <source_name> external sources
$ dbt run-operation stage_external_sources --args "select: <source_name>"

Expected results

1 of X START external source <schema>.<table>
....

Actual results

No external sources selected

Screenshots and log output

System information

The contents of your packages.yml file:

packages:
  - package: dbt-labs/dbt_utils
    version: 1.3.0
  - package: dbt-labs/codegen
    version: 0.12.1
  - package: dbt-labs/dbt_external_tables
    version: 0.10.0

Which database are you using dbt with?

I only tested Redshift, but since the change was in macros/common/stage_external_sources.sql it probably affects all databases.

The output of dbt --version:

Core:
  - installed: 1.8.8
  - latest:    1.8.9 - Update available!

Plugins:
  - postgres: 1.8.2 - Up to date!
  - redshift: 1.8.1 - Up to date!

The operating system you're using: Windows 11 Enterprise

The output of python --version: Python 3.10.15

Additional context

kyleburke-meq commented 23 hours ago

can confirm I am having the same issue on 0.10.0