opendatahub-io-contrib / data-mesh-pattern

Data Mesh Pattern
https://opendatahub-io-contrib.github.io/data-mesh-pattern
Apache License 2.0
28 stars 15 forks source link

Integrate Trino ETL Jobs using dbt-trino with Airflow on Kubernetes #85

Open caldeirav opened 1 year ago

caldeirav commented 1 year ago

Once a DBT pipeline has been set, we want to encapsulate the execution into a DAG on Airflow, running on OpenShift. Look at implementing this from a pipeline-as-code perspective (everything included in the pipeline repo).

Good reference article: https://itnext.io/the-way-to-integrate-trino-etl-jobs-using-dbt-trino-with-airflow-on-kubernetes-51cc851a366