This is my second attempt at addressing the issue that describe extended can truncate complex types. With the release of dbt-core 1.8.7, we can now process behavior flags; this PR introduces the choice of using information_schema for grabbing column information in get_columns_for_relation. I'm hiding behind a behavior flag because given the current state of UC information_schema, we have to run repair to trust that columns of recently created or altered table will be present in the information_schema, which adds overhead. Furthermore, this trick only works for Delta tables at this time. It is hoped that in time the sync issue with information_schema will be solved, but in the mean time, users can use this flag when they have complex types that describe extended truncates.
Checklist
[x] I have run this code in development and it appears to resolve the stated issue
[x] This PR includes tests, or tests are not required/relevant for this PR
[x] I have updated the CHANGELOG.md and added information about my change to the "dbt-databricks next" section.
Partial fix for #779
Description
This is my second attempt at addressing the issue that describe extended can truncate complex types. With the release of dbt-core 1.8.7, we can now process behavior flags; this PR introduces the choice of using information_schema for grabbing column information in get_columns_for_relation. I'm hiding behind a behavior flag because given the current state of UC information_schema, we have to run repair to trust that columns of recently created or altered table will be present in the information_schema, which adds overhead. Furthermore, this trick only works for Delta tables at this time. It is hoped that in time the sync issue with information_schema will be solved, but in the mean time, users can use this flag when they have complex types that describe extended truncates.
Checklist
CHANGELOG.mdand added information about my change to the "dbt-databricks next" section.