dbt-labs / dbt-spark

dbt-spark contains all of the code enabling dbt to work with Apache Spark and Databricks
https://getdbt.com
Apache License 2.0
405 stars 227 forks source link

[CT-2242] [CT-2227] [Bug] dbt Classes Instance Methods is_string, is_numeric, is_number, is_float don't seem to work #658

Closed mvdwielen closed 1 year ago

mvdwielen commented 1 year ago

Is this a new bug in dbt-core?

Current Behavior

The col.data_type prints int and string but calling col.is_string() returns false when data_type is string and col.is_numeric() returns false when data_type is int.

Expected Behavior

is_string, is_numeric, is_number, is_float should return false or true when data_type of column is a string, number, decimal or float

Steps To Reproduce

dbt with databricks sql warehouse

{% for col in cols -%} {{ log("Col: " ~ col.data_type, info=true) }} {{ log("Col: " ~ col.is_string() ~ col.is_numeric() ~ col.is_number() ~ col.is_float(), info=true) }} {%- endfor %}

image

Relevant log output

No response

Environment

- OS:
- Python:
- dbt:

Which database adapter are you using with dbt?

spark

Additional Context

No response

dbeatty10 commented 1 year ago

Thanks for reporting this @mvdwielen !

I was able to reproduce something similar.

Here's what I used with dbt-databricks:

seeds/my_seed.csv

my_string,my_numeric,my_number,my_float
US,123,456,7.89

analyses/get_columns.sql

{%- set cols = adapter.get_columns_in_relation(ref("my_seed")) -%}

{% for col in cols -%}
{{ log("Col: " ~ col, info=true) }}
{{ log("Col: " ~ col.name, info=true) }}
{{ log("Col: " ~ col.data_type, info=true) }}
{{ log("  is_string: " ~ col.is_string(), info=true) }}
{{ log("  is_numeric: " ~ col.is_numeric(), info=true) }}
{{ log("  is_number: " ~ col.is_number(), info=true) }}
{{ log("  is_float: " ~ col.is_float(), info=true) }}
{{ log("", info=true) }}
{%- endfor %}

Then I ran:

dbt seed
dbt compile -s analyses/get_columns.sql  

And got this output:

Col: <DatabricksColumn my_string (string)>
Col: my_string
Col: string
  is_string: False
  is_numeric: False
  is_number: False
  is_float: False

Col: <DatabricksColumn my_numeric (bigint)>
Col: my_numeric
Col: bigint
  is_string: False
  is_numeric: False
  is_number: True
  is_float: False

Col: <DatabricksColumn my_number (bigint)>
Col: my_number
Col: bigint
  is_string: False
  is_numeric: False
  is_number: True
  is_float: False

Col: <DatabricksColumn my_float (double)>
Col: my_float
Col: double
  is_string: False
  is_numeric: False
  is_number: False
  is_float: False
github-actions[bot] commented 1 year ago

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

github-actions[bot] commented 1 year ago

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.