generate_source doesn't get table names if `table_names` is none

VDFaller commented 10 months ago

Describe the bug

When running

dbt run-operation generate_source --args '{schema_name: my_schema, database_name: my_database}'

It just returns

sources:
  - name: my_schema
    database: my_database
    tables:

it does create them if I specify them explicitly with table_names, but that kind of defeats the point.

Expected results

sources:
  - name: my_schema
    database: my_database
    tables:
      - name: a
      - name: list
      - name: of
      - name: tables

Actual results

sources:
  - name: my_schema
    database: my_database
    tables:

Screenshots and log output

System information

The contents of your packages.yml file:

  - package: calogica/dbt_expectations
    version: 0.8.5
  - package: dbt-labs/dbt_utils
    version: 1.1.1
  - package: dbt-labs/spark_utils
    version: 0.3.0
  - package: dbt-labs/codegen
    version: 0.11.0

Which database are you using dbt with? databricks

The output of dbt --version: Core:

installed: 1.5.4
latest: 1.6.6 - Update available!

Your version of dbt-core is out of date! You can find instructions for upgrading here: https://docs.getdbt.com/docs/installation

Plugins:

databricks: 1.5.5 - Update available!
spark: 1.5.1 - Update available!

The operating system you're using: windows 11 The output of python --version: Python 3.11.4

Additional context

It looks like it might be due to codegen.get_tables_in_schema.

Are you interested in contributing the fix?

I'd love to help if I can figure out your contributing guidelines. I'll try and update if I need help.

VDFaller commented 10 months ago

Okay I assume this is a problem with spark_utils because if I do.

dbt run-operation generate_source --args '{"schema_name": "my_schema", "database_name": "my_database", "table_pattern": ""*""}'

It gives me back tables, but for my target.database, not my given database (the same schema is in both).

They clearly know they need to override the '%' because they already do it

I'll make an issue with them.

github-actions[bot] commented 4 months ago

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

github-actions[bot] commented 3 months ago

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

dbt-labs / dbt-codegen