wait_for_completion not working when set to True while triggering child dag from parent dag

[vashisht33]

Apache Airflow version

2.9.1

If "Other Airflow 2 version" selected, which one?

No response

What happened?

wait_for_completion not working when set to True while triggering child dag from parent dag.Below i have shared two files,one is parent_dag and other is child_dag.All i am trying to do is to trigger child dag from parent dag and wait for completion of child dag before i proceed further in parent_dag(to next task in line).But my child_dag gets in queued state and then scheduler starts skipping heartbeat,although there is no issue in the code as i was able to trigger child dag manually.Also,when i set wait_for_completion = False,it ran fine.

What you think should happen instead?

No response

How to reproduce

parent_dag.py from airflow import DAG from airflow.operators.dagrun_operator import TriggerDagRunOperator from airflow.operators.python_operator import PythonOperator from datetime import datetime, timedelta

default_args = { 'owner': 'airflow', 'depends_on_past': False, 'start_date': datetime(2024, 5, 27), 'email_on_failure': False, 'email_on_retry': False, 'retries': 1, }

parent_dag = DAG( 'parent_dag', default_args=default_args, schedule_interval='@daily', catchup=False )

def c(): print("c")

trigger_target = TriggerDagRunOperator( task_id='trigger_target', trigger_dag_id='child_dag', reset_dag_run=True, wait_for_completion=True, poke_interval=10, dag = parent_dag ) task_after_child_dag_completion = PythonOperator( task_id = "task_after_child_dag_completion", python_callable = c, dag = parent_dag )

trigger_target >> task_after_child_dag_completion

child_dag.py from airflow import DAG from airflow.operators.python_operator import PythonOperator from datetime import datetime, timedelta import time default_args = { 'owner': 'airflow', 'depends_on_past': False, 'start_date': datetime(2024, 5, 27), 'email_on_failure': False, 'email_on_retry': False, 'retries': 1, }

child_dag = DAG( 'child_dag', default_args=default_args, schedule_interval=None, catchup=False )

def child_task(): time.sleep(15) print("Executing child task")

child_task = PythonOperator( task_id='child_task', python_callable=child_task, dag=child_dag )

child_task

Operating System

Ubuntu 22.04.4 LTS

Versions of Apache Airflow Providers

No response

Deployment

Other

Deployment details

local env

Anything else?

No response

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[raphaelauv]

same than https://github.com/apache/airflow/issues/38353

if you agree, you can close this issue, thanks

[vashisht33]

What is the solution here? or expected timeline for bug fix? @potiuk @raphaelauv

[potiuk]

No - it's just duplicate issue. And solution will be there when someone fixes it. This is open-source project and most certain way to be sure on "expected" timeline is to prepare a PR fixing it. Otherwise it will have to be picked by someone who will decide to fix it.

[vashisht33]

OK,can you suggest some other solution if any @potiuk .I tried using externaltasksensor as well and they also gave same issue.

[potiuk]

OK,can you suggest some other solution if any @potiuk .I tried using externaltasksensor as well and they also gave same issue.

No i Have no suggestions. I have not looked into it.

[vashisht33]

can you tag someone else who might know the alternate solution? @potiuk

[potiuk]

can you tag someone else who might know the alternate solution? @potiuk

Nope. I have no idea whom I can tag. But what I suggest you is ypu can ask in Slack, open A girhub discussion about it, maybe even look in stack overflow, also you can check Ask Astro.