Closed NickYadance closed 2 years ago
@NickYadance I am running into this exact same issue. Has this been fixed in a later version?
No and the issue description is not actually accurate. The statistics query
mainly indicates that the query has been waiting
for a long time for something, not 100% deadlock between transactions, maybe due to slow query, high mysql server IO ...
For me , i resolve this by optimizing the IO load of the mysql server. For the daedlock, maybe you can find some help here #27000 with enough debugging info.
Apache Airflow version
Other Airflow 2 version
What happened
Scheduler got crashed soon after reboot.
Database engine status shows that some cycle dependency exists between 2 transactions, the select and update query towards table
task_instance
. And each time the scheduler reboot, the update transaction will timeout and causing scheduler crash.Noticed that the select query named with
airflow statistics
so i tried shutdown statsd (delete k8s deployments) blindly and it surprisingly worked. After that the scheduler reboot and run normally.What you think should happen instead
No response
How to reproduce
No response
Operating System
NAME="Ubuntu" VERSION="20.04.1 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.1 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
Code of Conduct