[Bug] Support Vector data types for macro `snowflake__get_columns_in_relation`

chamilton61 commented 4 months ago

Is this a new bug in dbt-snowflake?

[X] I believe this is a new bug in dbt-snowflake
[X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When trying to run an incremental load referencing a table that has a column that is cast as a VECTOR data type, I get the following error: Could not interpret data_type "VECTOR(FLOAT, 1536)": could not convert "FLOAT" to an integer

This appears to be due to the from_description Column function here expecting only integers within parenthesis.

Expected Behavior

Expected column data type to be properly parsed and reproduced in the incremental table.

Steps To Reproduce

do dbt run on a model that has an incremental load materialization and has a column that is the VECTOR data type

Relevant log output

15:40:11  Snowflake adapter: Error running SQL: macro get_columns_in_relation
15:40:11  Snowflake adapter: Rolling back transaction.
15:40:11  On <model name>: Close
15:40:11  Runtime Error in model <model name> (<model name>.sql)
  Could not interpret data_type "VECTOR(FLOAT, 1536)": could not convert "FLOAT" to an integer

Environment

- OS: Ubuntu 22.04.4 LTS
- Python:  3.11.7
- dbt-core: 1.8.3
- dbt-snowflake: 1.8.3

Additional Context

suspected cause of the issue: https://github.com/dbt-labs/dbt-snowflake/blob/2576a08379e248d393b9a698e58fac0d32c23648/dbt/include/snowflake/macros/adapters.sql https://github.com/dbt-labs/dbt-adapters/blob/main/dbt/adapters/base/column.py#L130

cgarretty commented 4 weeks ago

+1 - we're also being impacted by this.

brandonpeebles commented 3 weeks ago

+1 — also been dealing with this error trying to reference source iceberg tables that contain Array or Object data types.

dbt-labs / dbt-snowflake