search

astronomer / astronomer-cosmos

Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code
https://astronomer.github.io/astronomer-cosmos/
Apache License 2.0
573 stars 142 forks source link

Model selection "plus" operator not properly parsed when selecting tags #823

Closed enkeboll closed 1 week ago

enkeboll commented 6 months ago

Migrating an issue raised in the #airflow-dbt slack org: image image

Basically, the "plus" operator isn't properly parsed when used with tags. So tag:every-hour works but +tag:every-hour or tag:every-hour+ result in empty graphs. We use this syntax in our DBT Cloud scheduled jobs, even though it isn't explicitly called out in the DBT docs: image

I'm happy to try to take a stab at adding this functionality to selector.py, unless there's a reason to hold off.

dosubot[bot] commented 6 months ago

🤖

That's a great initiative! Your willingness to take a stab at adding this functionality to selector.py is much appreciated. Your contribution would definitely be valuable to the community. Thank you for considering this and feel free to reach out if you need any guidance along the way.

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

jbandoro commented 6 months ago

Thanks @enkeboll for opening this issue!

I'm happy to try to take a stab at adding this functionality to selector.py, unless there's a reason to hold off.

That would be great, thanks! Let us know if you need any support.

danielvdende commented 4 months ago

Any update on this? We just hit the same issue 🥲

enkeboll commented 4 months ago

I personally haven't started work on this yet, though it's still in my backlog.

tatiana commented 3 months ago

@enkeboll @danielvdende, would you have bandwidth to work on this?

gbatiz commented 3 months ago

Would it be possible to for the development team to pick this up?

We are making use of custom node renderers for sources (sensor to wait on loading task) and exposures (operator triggering refresh in BI-tool), but schedule-related tags only make sense in case models.

We would like to run models tagged daily in a daily Airflow DAG, models tagged weekly in a weekly DAG, etc. However, without properly functioning graph selectors, we cannot include sources and exposures for these models.

We have quite extensive workarounds for now to make this work, but would like to get rid of them.

tatiana commented 1 week ago

It seems @mvictoria is solving the issue in #1150 🎉