astronomer / astronomer-cosmos

Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code
https://astronomer.github.io/astronomer-cosmos/
Apache License 2.0
576 stars 144 forks source link

[Bug] Cosmos `dbt run` inside DbtDag/DbtTaskGroup can select multiple nodes #1093

Open dwreeves opened 1 month ago

dwreeves commented 1 month ago

Astronomer Cosmos Version

Other Astronomer Cosmos version (please specify below)

If "Other Astronomer Cosmos version" selected, which one?

1.5.0

dbt-core version

1.8.0

Versions of dbt adapters

dbt-snowflake==1.8.3

LoadMode

DBT_LS_MANIFEST

ExecutionMode

LOCAL

InvocationMode

DBT_RUNNER

airflow version

2.9.2

Operating System

not applicable

If a you think it's an UI issue, what browsers are you seeing the problem on?

No response

Deployment

Astronomer

Deployment details

No response

What happened?

If you have a project structure that looks something like this:

dbt/
| - models/
  | - foo/
  | - a.sql
  | - foo.sql

Then the node selected for foo will run both a and foo.

The solution is most likely to use the fqn to select nodes instead of just the node name to guarantee uniqueness: https://docs.getdbt.com/reference/node-selection/methods#the-fqn-method


Also a note, LoadMode.DBT_MANIFEST is the proper name for the load mode, not DBT_LS_MANIFEST 🤷 This should be fixed in the bug report issue template. Also, the only version you can select of Cosmos is 1.4.1.

Relevant log output

No response

How to reproduce

See above

Anything else :)?

No response

Are you willing to submit PR?

Contact Details

No response

tatiana commented 1 month ago

Thanks for reporting this, @dwreeves !

Do you think the fqn method would also allow us to continue supporting versions? This was the one concern that popped up, thinking about the changes we had in selecting nodes in the past months