search

dagster-io / dagster

An orchestration platform for the development, production, and observation of data assets.
https://dagster.io
Apache License 2.0
11.36k stars 1.44k forks source link

Enabling debug mode in dbt cli resource causes failure #13586

Closed pablo-statsig closed 1 year ago

pablo-statsig commented 1 year ago

Dagster version

dagster, version 1.2.6

What's the issue?

When I enable debug mode in the dbt_cli_resouce, I run into this failure when running a dbt asset job: dagster._core.errors.DagsterInvariantViolationError: Compute for op "run_dbt_1e50f_f83a8" returned an output "<model_name>" multiple times

This does not happen when I set the debug mode to False

What did you expect to happen?

For the job to not fail for a new reason.

How to reproduce?

My set up is a dbt project with multi partitions of [date, static partition]. I created an asset job with about 40 models in it. When I run the job it fails due to some SQL error but when I run it with debug mode it fails with a new error coming from dagster saying:

dagster._core.errors.DagsterInvariantViolationError: Compute for op "run_dbt_1e50f_f83a8" returned an output "<model_name>" multiple times

Deployment type

Dagster Helm chart

Deployment details

Dagster helm running on AKS

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

rexledesma commented 1 year ago

Hi @pablo-statsig, thanks for the report. This is resolved with https://github.com/dagster-io/dagster/pull/15905 when using DbtCliResource, and it will be released in our 1.4.7 release of Dagster.