Closed mvdwielen closed 1 year ago
Thanks for reporting this @mvdwielen !
I was able to reproduce something similar.
Here's what I used with dbt-databricks:
seeds/my_seed.csv
my_string,my_numeric,my_number,my_float
US,123,456,7.89
analyses/get_columns.sql
{%- set cols = adapter.get_columns_in_relation(ref("my_seed")) -%}
{% for col in cols -%}
{{ log("Col: " ~ col, info=true) }}
{{ log("Col: " ~ col.name, info=true) }}
{{ log("Col: " ~ col.data_type, info=true) }}
{{ log(" is_string: " ~ col.is_string(), info=true) }}
{{ log(" is_numeric: " ~ col.is_numeric(), info=true) }}
{{ log(" is_number: " ~ col.is_number(), info=true) }}
{{ log(" is_float: " ~ col.is_float(), info=true) }}
{{ log("", info=true) }}
{%- endfor %}
Then I ran:
dbt seed
dbt compile -s analyses/get_columns.sql
And got this output:
Col: <DatabricksColumn my_string (string)>
Col: my_string
Col: string
is_string: False
is_numeric: False
is_number: False
is_float: False
Col: <DatabricksColumn my_numeric (bigint)>
Col: my_numeric
Col: bigint
is_string: False
is_numeric: False
is_number: True
is_float: False
Col: <DatabricksColumn my_number (bigint)>
Col: my_number
Col: bigint
is_string: False
is_numeric: False
is_number: True
is_float: False
Col: <DatabricksColumn my_float (double)>
Col: my_float
Col: double
is_string: False
is_numeric: False
is_number: False
is_float: False
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.
Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.
Is this a new bug in dbt-core?
Current Behavior
The col.data_type prints int and string but calling col.is_string() returns false when data_type is string and col.is_numeric() returns false when data_type is int.
Expected Behavior
is_string, is_numeric, is_number, is_float should return false or true when data_type of column is a string, number, decimal or float
Steps To Reproduce
dbt with databricks sql warehouse
{% for col in cols -%} {{ log("Col: " ~ col.data_type, info=true) }} {{ log("Col: " ~ col.is_string() ~ col.is_numeric() ~ col.is_number() ~ col.is_float(), info=true) }} {%- endfor %}
Relevant log output
No response
Environment
Which database adapter are you using with dbt?
spark
Additional Context
No response