brooklyn-data / dbt_artifacts

A dbt package for modelling dbt metadata. https://brooklyn-data.github.io/dbt_artifacts
Apache License 2.0
309 stars 113 forks source link

[Bug] Had an issue upgrading dbt-artifact to 2.6.0 #397

Closed ajonesINQ closed 4 months ago

ajonesINQ commented 8 months ago

Overview

Tried upgrading dbt-artifacts from 2.4 to 2.6. the upgrade was successful, however after our job ran the artifact was not able to be uploaded to bigquery as the query was too long.

How to reproduce

Expected behaviour

We would expect the artifact data to be uploaded to bigquery and the job not to fail

Screenshots

job_fail_for_issue

Environment

Results of running dbt --version: dbt=1.6.5

# Add the results here

Please paste the contents of your packages.yml file here:

packages:
  ------------------------------------------------------------
  Invoke dbt Command
------------------------------------------------------------
dbt deps

05:00:49  Running with dbt=1.6.5
05:00:49  Installing dbt-labs/dbt_utils
05:00:49  Installed from version 1.1.1
05:00:49  Installing calogica/dbt_expectations
05:00:50  Installed from version 0.10.0
05:00:50  Installing dbt-labs/codegen
05:00:50  Installed from version 0.11.0
05:00:50  Installing data-mie/dbt_profiler
05:00:50  Installed from version 0.8.1
05:00:50  Installing dbt-labs/dbt_project_evaluator
glsdown commented 8 months ago

Hello. As noted in #369 we added a new feature in 2.6.0 which will hopefully help here. In a previous release, we added an additional all_results column which contained a JSON object of all results for people to be able to do their own analysis on. However, this increased the query size dramatically, which is why these issues started arising.

In the latest release, I've added a variable that will stop this behaviour to reduce the query size. Please can you try adding the following as a variable in your dbt_project.yml file and see if that resolves it?

vars:
  dbt_artifacts_exclude_all_results: true
jared-rimmer commented 5 months ago

@ajonesINQ are you still seeing this issue after implementing the above?

data-modeler commented 4 months ago

I came here with the same problem and adding the variable solved the problem for me. Thanks!

ajonesINQ commented 4 months ago

So sorry thought I commented on this ages ago the above did resolve my issue. thank you