eonezhang / mysql-master-ha

Automatically exported from code.google.com/p/mysql-master-ha
0 stars 0 forks source link

Event scheduler can block the failover #28

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago
What steps will reproduce the problem?

When event scheduler is activated, a process can block the failover due that 
the manager consider that like a long update.

What is the expected output?

You should ignore the event scheduler thread in the failover process.
It's possible to bypass this issue with a "set global event_scheduler=OFF" 
command on the mysql servers(Master and slaves)

What do you see instead?

Tue Jun 12 11:27:56 2012 - [info] Executing FLUSH NO_WRITE_TO_BINLOG TABLES. 
This may take long time..
Tue Jun 12 11:27:56 2012 - [info]  ok.
Tue Jun 12 11:27:56 2012 - [info] Checking MHA is not monitoring or doing 
failover..
Tue Jun 12 11:27:56 2012 - [info] Checking replication health on XXX.XX.XX.XX..
Tue Jun 12 11:27:56 2012 - [info]  ok.
Tue Jun 12 11:27:56 2012 - [error][/.../MasterRotate.pm, ln161] We should not 
start online master switch when one of connections are running long updates on 
the current master. Currently 1 update thread(s) are running.
Details:
{'Time' => '48476','Command' => 'Daemon','db' => undef,'Id' => '1','Info' => 
undef,'User' => 'event_scheduler','State' => 'Waiting for next 
activation','Host' => 'localhost'}
Tue Jun 12 11:27:56 2012 - [error][/.../ManagerUtil.pm, ln178] Got ERROR:  at 
/.../masterha_master_switch line 53

What version of the product are you using? On what operating system?

0.53
Debian 6

Original issue reported on code.google.com by cedric.p...@gmail.com on 12 Jun 2012 at 4:36

GoogleCodeExporter commented 8 years ago
I'm also having the same problem.  However I would like to add that as far as I 
can tell there is no option to handle the failover of events to a new master.

In the case that you have your events set to SLAVESIDE_DISABLED (see the 
information_schema.events table) and have the event scheduler running on all 
servers, the slave that becomes that master will not start running the events 
because the id of the originator has not changed 

In the case that you have your events set to ENABLED (see the 
information_schema.events table) and have the event scheduler running only on 
the master server, the event scheduler will not be enabled on the slave that 
gets promoted to the new master.

The plan to get around this for the time being by adding something in our 
master_ip_failover_script to enable the event scheduler after re-assigning the 
VIP.  This way in the case of failover that results from system failure the 
event scheudle will be enabled.  In the case that it's a planned failover we'll 
just manipulate the event schedulers on the servers directly.  Admittedly this 
is a bit janky and it would be better to have something in the MHA solution to 
handle this.

Original comment by peter.t....@gmail.com on 7 May 2014 at 2:35