search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
35.62k stars 13.87k forks source link

Add task status filters to Task Duration Bar chart view #40445

Open ketozhang opened 4 weeks ago

ketozhang commented 4 weeks ago

Description

Currently the Task Duration bar chart shows every task instance. The only filter available are DAG type and state, but there is no way to filter task-level metadata.

Use case/motivation

For example, adding a task status filter, we can filter out any task that were skipped.

Here is what it currently looks like with a DAG that skip this task more often than not...

image You must agree the histogram is not so useful with all these skipped tasks.

Related issues

No response

Are you willing to submit a PR?

Code of Conduct

tirkarthi commented 3 weeks ago

Currently the chart just takes the task instance row and plot the duration. Maybe something like filters for the overall grid will make this easier for the plot.

Related : https://github.com/apache/airflow/issues/38032

@bbovenzi

bbovenzi commented 1 week ago

I agree. The grid view is the main interface to navigate around a DAG. Let's add filters to that and have the task duration page use those filters.

As I wrote here, we could also combined the task and run duration pages and included chart-specific filters too

ketozhang commented 1 week ago

100% agree with both. Will keep the scope of this ticket to the histogram view, but definitely would be much better to have the filter all viz in the Grid view.