mara / mara-pipelines

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
MIT License
2.07k stars 100 forks source link

Better monitoring of pipeline executions [EPIC] #82

Open leo-schick opened 2 years ago

leo-schick commented 2 years ago

User story

Currently the only way to see in the UI if a pipeline run through is to click on the pipeline and to check its last runs. There you can see if the pipeline still runs (unfinished), succeeded or failed. When you have a bunch of pipelines (e.g. in the root pipeline) which run on different time schedules, you have to click through all of them to know if everything worked out fine.

This shall be improved

Ideas how to solve this