search

astronomer / astronomer-cosmos

Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code
https://astronomer.github.io/astronomer-cosmos/
Apache License 2.0
520 stars 132 forks source link

Link to dbt docs from task #1012

Open dwreeves opened 1 month ago

dwreeves commented 1 month ago

Description

We should be able to use this: https://airflow.apache.org/docs/apache-airflow/stable/howto/define-extra-link.html to link to dbt docs from the task instance view in the UI. This will require a few things that could be tricky:

Use case/motivation

No response

Related issues

No response

Are you willing to submit a PR?

dwreeves commented 1 month ago

Setting this as a 1.6 milestone. I played around with this and, well, there are a lot of extra complications in making this work aside from just multiple dbt projects. 😬 TLDR, what you really need is the information contained in the "fully qualified name" (fqn) of a node to link it to the docs properly, and we just don't really use that information right now in Cosmos. Most notably, the project name is not able to be inferred based on the construction of an operator. (The node type can be inferred from the base_cmd, and the node itself can be inferred from the --select / --models.) So this would require passing new information into the operator somehow.

There are a few more issues. One big one is that an external link is not nullable / cannot be disabled for an operator. This is an issue for Operators that do not tie to single nodes.