dbt-labs / dbt-codegen

Macros that generate dbt code
https://hub.getdbt.com/dbt-labs/codegen/latest/
Apache License 2.0
464 stars 102 forks source link

[CT-1605] [Bug] generate_source sometimes fails without explanation #95

Closed jaredx435k2d0 closed 1 year ago

jaredx435k2d0 commented 1 year ago

Is this a new bug in dbt-snowflake?

Current Behavior

If I run dbt run-operation generate_source --args '{"schema_name": "my_schema_name", "database_name": "my_db_name"}'

The output will sometimes be:


version: 2

sources:
  - name: my_schema_name
    database: my_db_name
    tables:

Other times, it works great.

Expected Behavior

I would expect a full sources document to be created, with each table, always.

Steps To Reproduce

It consistently happens with the same schemas, but I'm not sure why. Run dbt run-operation generate_source --args '{"schema_name": "my_schema_name", "database_name": "my_db_name"}' See that output never gets to the first table.

Both schemas for which I'm currently seeing this problem are populated via Fivetran's Google Sheets connector, and have _ROW as their first column

Still, I have other schemas populated by the same connector (in the same DB), with tables that have the same first column name, that work great with this command.

Relevant log output

I don't see any logs for this, nor error messages.

Environment

- OS: macOS 12.6.1 (21G217)
- Python: Python 3.10.7
- dbt-core:
- dbt-snowflake: 1.3.0

Additional Context

No response

dbeatty10 commented 1 year ago

@jaredx435k2d0 this looks like it's related to the generate_source macro within the codegen package, so I'm going to transfer this issue over there for its maintainers to troubleshoot further.

If the root cause(s) end up being in the dbt-snowflake repo, then they can transfer this issue back again.

boblannon-fp commented 1 year ago

this looks related to my issue, i think... under "additional context" I included my findings. It seems to have to do with the fact that snowflake returns column names in all-caps.

https://github.com/dbt-labs/dbt-codegen/issues/82

github-actions[bot] commented 1 year ago

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

github-actions[bot] commented 1 year ago

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.