Add job schedulers and orchestrators for data pipelines - Githubissues

brnaguiar / mlops-next-watch

MLOps project that recommends movies to watch implementing Data Engineering and MLOps best practices.

Other

0 stars 1 forks source link

Add job schedulers and orchestrators for data pipelines #6

Closed brnaguiar closed 1 year ago

brnaguiar commented 1 year ago

For example Airflow:

Best practices:

https://towardsdatascience.com/apache-airflow-in-2022-10-rules-to-make-it-work-b5ed130a51ad

brnaguiar commented 1 year ago

https://github.com/cordon-thiago/airflow-spark/tree/master