brooklyn-data / dbt_artifacts

A dbt package for modelling dbt metadata. https://brooklyn-data.github.io/dbt_artifacts
Apache License 2.0
323 stars 119 forks source link

Add variable to allow users to exclude all results columns #382

Closed glsdown closed 1 year ago

glsdown commented 1 year ago

Overview

As can be seen in #369, there are issues where the BQ length gets too large. This has started happening since introducing this column to return all results available in the objects. Therefore, I have added a new variable dbt_artifacts_exclude_all_results which can be set to true to disable this column from being populated.

Ideally, we'd want the default to be to 'not' include this field, but as it's been released with the field included (i.e. opt-in), it doesn't feel right to do that, so I have built it as an opt-out feature.

Update type - breaking / non-breaking

What does this solve?

Outstanding questions

What databases have you tested with?

stevenkoppenol commented 1 year ago

Great proposal. I hope this will solve the "query too long" error.