[Bug/Feature] DBT does nothing on inexistent model

[taielc]

Context

​ By defalt dbt run --select "a_model_that_does_not_exist" will do nothing and only print a Warning:

WARNING: Nothing to do. Try checking your model configs and model specification args

​

Example

​ If one creates a dag with a single DbtRunOperator selecting "a_model_that_does_not_exist" it runs successfully, which I don't think should be the right behaviour:

from airflow_dbt_python.operators.dbt import DbtRunOperator
from airflow import DAG
​
with DAG(...) as dag:
    operator = DbtRunOperator(
        models=["this_does_not_exist"],
        ...
    )

​ I just get a "silently failing" successful task in airflow when run.

[taielc]

In the past I have solved this by parsing the outputs to search and get the WARNING message and raise an airflow failure exception.

[tomasfarias]

Thanks for opening an issue.

I disagree, airflow-dbt-python is consistent with dbt-core: a warning is not an error and I rather let dbt-core decide what is an error.

There is a solution if you wish to elevate warnings to errors and that is to set the warn_error argument to True. This argument works just as the --warn-error flag would work with dbt CLI. Personally, I suggest running all production workflows with warn_error=True.

In your example:

from airflow_dbt_python.operators.dbt import DbtRunOperator
from airflow import DAG
​
with DAG(...) as dag:
    operator = DbtRunOperator(
        models=["this_does_not_exist"],
        warn_error=True,
        ...
    )

That being said, this was not covered by our test cases, so I wrote one and it locally works as expected. Will open a PR for that soon.

[tomasfarias]

For context, this is the behavior of dbt CLI on a new test project:

$  dbt run --select "a_model_that_does_not_exist"
17:24:35  Running with dbt=1.4.5
17:24:35  Unable to do partial parsing because saved manifest not found. Starting full parse.
17:24:35  Found 2 models, 4 tests, 0 snapshots, 0 analyses, 291 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics
17:24:35  The selection criterion 'a_model_that_does_not_exist' does not match any nodes
17:24:35  
17:24:35  Nothing to do. Try checking your model configs and model specification args
$  echo $?
0
$  dbt --warn-error run --select "a_model_that_does_not_exist"
17:27:00  Running with dbt=1.4.5
17:27:00  Found 2 models, 4 tests, 0 snapshots, 0 analyses, 291 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics
17:27:00  Encountered an error:
Compilation Error
  The selection criterion 'a_model_that_does_not_exist' does not match any nodes
$  echo $?
2

Without a good reason, I rather airflow-dbt-python replicate the behavior that dbt users expect from the CLI.

[taielc]

Great! Thanks for the answer, I think I'll be using the flag as you suggested :)