search

dbt-labs / dbt-external-tables

dbt macros to stage external sources
https://hub.getdbt.com/dbt-labs/dbt_external_tables/latest/
Apache License 2.0
294 stars 119 forks source link

[BigQuery] Support for arbitrary REST API configurations #193

Closed akerone closed 11 months ago

akerone commented 1 year ago

Describe the feature

The BigQuery REST API is very well documented, and new features are being added at a rapid pace. Especially important to this project would be the ExternalDataConfiguration section. In my case I absolutely need to be able to set the connectionId field to create a BigLake table, but it feels weird to ask for something that specific when we're configuring the desired table in YML that could be effortlessly converted to a JSON for the REST API.

Making the external section of the YML identical to the ExternalDataConfiguration of the REST API for BigQuery tables would immediately make this project way more usable and better documented for any BigQuery users. It would also mean virtually zero maintenance work in this project for BigQuery from this point forward.

Describe alternatives you've considered

As I mentioned, the alternative is to implement each and every new feature in SQL, but that's extra work for this project's contributors every time a new feature is added.

Additional context

This request affects BigQuery only.

Who will this benefit?

This would benefit all BigQuery users, and offload a lot of maintenance and documentation work from this project.

github-actions[bot] commented 11 months ago

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

github-actions[bot] commented 11 months ago

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

thomas-vl commented 7 months ago

@jeremyyeo I do like this suggestion but it is a backwards incompatible change. What are your thoughts?