Links to dagruns beyond last 25 runs don't work (take 3)

[hterik]

Apache Airflow version

2.9.1

If "Other Airflow 2 version" selected, which one?

No response

What happened?

This is a continuation of https://github.com/apache/airflow/issues/34723 that was fixed in https://github.com/apache/airflow/pull/34887, later fixed again in https://github.com/apache/airflow/pull/37018, reported as problematic for other reasons in https://github.com/apache/airflow/issues/38939, then later fixed yet again in https://github.com/apache/airflow/pull/38941. Unfortunatelly the issue still remains.

In short, the problem is that the grid view by default only shows 25 dagruns, with base_date starting at current time. If dag_run_id query parameter points to a dag run that is older than today - 25 runs, it will not be pre-selected. This makes it impossible to permalink to specific runs, without knowing in advance the base_date for that run.

1.Start 50 instances of dag mydag and wait for them to complete

2. Visit Browse -> Task Instances list: airflow/taskinstance/list/?_flt_3_dag_id=mydag
3. Scroll to the bottom and click on the oldest run, in the Task Id column, this link will look something like /dags/mydag/grid?task_id=mytask&dag_run_id=123456&tab=graph
4. Dag grid view opens, but no dagrun is selected. :slightly_frowning_face:

What you think should happen instead?

Dag grid should open and have dag_run_id=123456 selected.

It appears like there are two different endpoints, one called /graph, that does handle redirecting the base_date to include older runs, and another called /grid that doesn't. When browsing from the Browse -> DAG runs list instead, it uses the /graph endpoint and there the links work, even for older runs. :+1: @bbovenzi, do you know if the /graph is the one to use for all permalink purposes and this solves these base_date issues?

How to reproduce

-

Operating System

Debian Bookworm

Versions of Apache Airflow Providers

No response

Deployment

Other Docker-based deployment

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[RNHTTR]

There's some additional bizarre behavior that might be related. When clicking on the logs icon from Browse -> Task Instances, it only shows a single DAG run (the only one that was scheduled -- the other ~40 were triggered by button mashing), and it says "No XCom" where the logs should be:

[mpolatcan]

Hi @hterik, we faced same issue too. Problem is that task_instance.log_url generates log url to page that has this error. So, I resolved issue by generating custom execution log url with below function and use in operator callbacks for our Slack notifications. Basically, function generate url to see full execution log, not routes traffic to grid view 😅:

from datetime import datetime
from urllib.parse import quote

def __generate_execution_log_url(context: dict):
    dag_id = context["dag_run"].dag_id
    task_instance = context["ti"]
    task_id = task_instance.task_id
    encoded_execution_date = quote(task_instance.execution_date.strftime("%Y-%m-%dT%H:%M:%S.%f%z"))

    return f"https://airflow.example.com/log?dag_id={dag_id}&task_id={task_id}&execution_date={encoded_execution_date}"

[sbrinkhorst]

I have built the main branch of the project (at commit https://github.com/apache/airflow/commit/c5c50cc07f6fcd704981139beb54095d8b9938c7 to be precise) and still see the problem. Should we reopen this ticket or should I create a new one? Or is there something I'm missing here?