search

rocs-org / data-pipelines

Data engineering monorepo with Airflow, dbt and postgres
https://ornt.biologie.hu-berlin.de/airflow
4 stars 1 forks source link

Refactor airflow dbt task to run dbt models as individual airflow tasks #225

Closed jakobkolb closed 1 year ago

jakobkolb commented 1 year ago

Fix #224

netlify[bot] commented 1 year ago

Deploy Preview for rocs-dbt-docs canceled.

Name Link
Latest commit bc8490178b86161d4e06543a8edcbd2f33afe760
Latest deploy log https://app.netlify.com/sites/rocs-dbt-docs/deploys/63c7eff3900fa9000a674174
davhin commented 1 year ago

Nice work! I'm curious to see what happens to the runtime of dbt tasks if we have more ramp up and ramp down from Airflow included but this should not be a concern at this stage and the added visibility will be awesome!