update external table columns

dbt-labs / dbt-external-tables

dbt macros to stage external sources

https://hub.getdbt.com/dbt-labs/dbt_external_tables/latest/

Apache License 2.0

288 stars 116 forks source link

update external table columns #252

Closed thomas-vl closed 3 months ago

thomas-vl commented 5 months ago

Description & motivation

resolves: #263 Descriptions and policyTags on columns where not propagated to the table schema of an external table on BigQuery.

Checklist

[x] I have verified that these changes work locally
[x] I have updated the README.md (if applicable)
[x] I have added an integration test for my fix/feature (if applicable)

thomas-vl commented 5 months ago

@jeremyyeo do you have any idea why CI fails on snowflake and redshift?

thomas-vl commented 5 months ago

@jeremyyeo CI is broken see the PR I created without any changes: https://github.com/dbt-labs/dbt-external-tables/pull/253

thomas-vl commented 4 months ago

@dataders I saw you merge another pr, could you also take a look at this one?

dataders commented 3 months ago

@thomas-vl thinking about this again. Ultimately, when External Tables are in Core, implementing your proposed change will be both easier and more robust. However that is still a ways away.

I'm comfortable accepting this PR given that:

this shouldn't affect non-BQ users, and
it's a stop-gap solution