Mapped Task details view "success" counts are off

[jsnb-devoted]

Apache Airflow version

2.5.1

What happened

After the upgrade to 2.5.1 we have noticed that the counts in the details page for mapped tasks seems to be wrong. The "success" count is typically stuck at 1 and then if you look at the actual tasks there are many successful tasks.

What you think should happen instead

Here is the details page saying there is 1 successful task: In the "Mapped Tasks" view you can see there are many successful tasks

How to reproduce

Run a DAG with mapped tasks -- watch the success count

Operating System

Ubuntu

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==7.2.1
apache-airflow-providers-cncf-kubernetes==5.2.1
apache-airflow-providers-common-sql==1.3.3
apache-airflow-providers-datadog==3.1.0
apache-airflow-providers-facebook==3.1.0
apache-airflow-providers-ftp==3.3.1
apache-airflow-providers-http==4.2.0
apache-airflow-providers-imap==3.1.1
apache-airflow-providers-jdbc==3.3.0
apache-airflow-providers-postgres==5.4.0
apache-airflow-providers-snowflake==4.0.4
apache-airflow-providers-sqlite==3.3.1
apache-airflow-providers-ssh==3.4.0

Deployment

Official Apache Airflow Helm Chart

Deployment details

k8s on EKS with the official helm chart 1.8.0

Anything else

No response

Are you willing to submit PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[bbovenzi]

Is auto-refresh active on the page? If you refresh the page yourself does the issue persist?

[jsnb-devoted]

Is auto-refresh active on the page? If you refresh the page yourself does the issue persist?

@bbovenzi yep auto-refresh is turned on and I tried manually refreshing but the issue persisted.

[bbovenzi]

I have yet to replicate this issue.

Interesting how the count is also different from the table of mapped tasks. Do you know which one is the correct count?

[jsnb-devoted]

@bbovenzi -- count and success are both wrong. running queued scheduled and up for retry look closer to the right numbers but I didn't scrutinize them.

[bbovenzi]

I just replicated this issue. Hope to figure out a fix soon.

[jsnb-devoted]

Thanks so much @bbovenzi -- I hadn't had time to dig into the code or the meta db to diagnose the issue. Appreciate you doing the legwork here.

[vvnekb]

It affects us too, mapped tasks counters are off nearly all the time. Success counter and total mapped tasks counter are incorrect. We are using version 2.7.1.

[leodrivera]

We are using version 2.7.1 and are also being affected.

[attilaszombati]

We are using v2.5.3 and the the bug is here as well

[josephangbc]

We are using v2.7.3 and encountered this issue as well

[bbovenzi]

I know I replicated this a while ago, but then got distracted. Could anyone please provide an example dag that this happens with consistently?

[carlospalol]

Seeing this in Airflow 2.9.1.

I cannot provide an example DAG at the moment. It is happening in a task mapping that lives inside a task group. This task mapping currently has 86 tasks. 63 are successful, 3 are running, and 20 are scheduled.

Note the incorrect total "27 Tasks Mapped" and "success: 4".

Some of the successful tasks had retries.

The Task Instance List page shows all tasks fine, with the correct status and the correct number of items for each.

After some progress:

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

[github-actions[bot]]

This issue has been closed because it has not received response from the issue author.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

[bbovenzi]

We're going to have to refactor the grid_data endpoint and auto-refresh in Airflow 3 and with dag versioning. We should be able to fix this at the same time.