jgreben
commented
6 days ago I think because the weighting os on a per-task basis, it it probably scoped to its own dag and not across dags, even though the polling dag is triggered by the D-B-instances dag.
When running the dags in airflow-stage, it looks like all the running slots in the pool get taken up by the polling dag. This screenshot shows 14 running task for the polling dag and 16 tasks in the digital_bookplate_instances DAG queued.
The priority weight of the poling dag tasks are 3 while the ones for digital_bookplate_instances are queued with priority weight 5. I thought higher number means more priority. 🤷 https://airflow.apache.org/docs/apache-airflow/2.8.3/administration-and-deployment/priority-weight.html#priority-weights