search

z3z1ma / dbt-osmosis

Provides automated YAML management, a dbt server, streamlit workbench, and git-integrated dbt model output diff tools
https://z3z1ma.github.io/dbt-osmosis/
Apache License 2.0
422 stars 45 forks source link

Support Documentation of Struct Fields in BigQuery #32

Closed brandon-segal closed 1 year ago

brandon-segal commented 1 year ago

Proposed Behavior

When a user runs dbt-osmosis yaml document it will also pull in and preserve struct level documentation of fields in bigquery projects.

Current behavior

When there are struct fields in the schema.yml or source.yml the process removes any struct documentation that is specified as a field like name: struct.struct_field

Specifications

riordan commented 1 year ago

This would be incredibly useful, particularly when building staging models that have infrequently-accessed, grouped fields. This can save massively on query costs while preserving context for folks who don't have to hunt around and perform the right join when they need it.

brandon-segal commented 1 year ago

@riordan Just added a PR #35, which primarily aims at addressing the struct fields by updating how dbt-osmosis retrieves the column names.

brandon-segal commented 1 year ago

@z3z1ma I am not sure what the contribution process is like but I'd love to use this tool and I'm sure many others who use BigQuery would also like to use it. Would love to get your thoughts on the PR #35